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Introduction 


by Patrizia Falzetti 


Data are precious resources and as such an asset of the community. Col- 
lected, ordered, studied data become a tool that allows us to open debates and 
make most useful decisions. The INVALSI Statistical Service followed up, 
on this premise, with the organization of the IV Seminar “INVALSI data: a 
tool for teaching and scientific research”. The Institute’s databases, in fact, 
allow researchers and teachers to investigate in depth the theme of education 
in schools from different points of view and the event allows them to partic- 
ipate in a lively confrontation on the subject. 

The authors of the contributions collected in this volume investigate in 
depth the characteristics of students, schools and society. School and society 
are the two social institutions outside home through which the individual 
begins its own socialization process. Furthemore, the school system plays 
a strategic role because it is where future citizens acquire knowledge and 
skills. Hence the importance of analyzing data to question and seek explana- 
tions and solutions to any critical issues in terms of learning. 

In the first three chapters the main topic is about students’ performance: 
in the first one, through spatial autocorrelation techniques, it is investigat- 
ed which individual characteristics can affect the results of the INVALSI 
English tests. In the second, the authors reflect on the existence of a close 
relationship between the results achieved by students and the involvement of 
parents in preparatory learning activities. In the third chapter, however, we 
read about an in-depth analysis of the learning outcomes detected through 
the INVALSI Grade 10 tests of the IeFP in Trentino. 

In the following three chapters the focus is mainly on the school. Ac- 
cording to ISTAT projections, the number of primary and secondary school 
students in our country is expected to drastically decrease in the next years 
while the share of non-native students will increase, the author conducts an 
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exploratory analysis. The aim is to reconstruct the geographic (provincial 
levels) and typological (with reference to the characteristics of individual 
schools) variability of any discriminatory assessment of foreign students 
(Chapter 4). 

Chapter five reports the first phase of a research project carried out in a 
primary school in Southern Italy and aimed at implementing an interim as- 
sessment system. The contribution also critically reflects on how to convey 
and facilitate extensive use of data obtained from standard assessments for 
educational purposes. Identifying national training needs within schools and 
proposing guidelines for the improvement of practices relating to the use 
of INVALSI surveys, are the topic of the sixth chapter. The volume closes 
with a wide reflection on the theme of education that starts from one of the 
objectives of Agenda 2030, specifically Goal 4 “Quality education for all”. 

The work shows how the integration of data from various sources 
(INVALSI data on skills, ISTAT data provided to the survey for moni- 
toring progress towards Sustainable Development and data from MIUR 
source) can give an overall picture of the situation in our country and 
allow us to intervene where necessary. 

As a Statistical Service we hope that the reading of the volume confirms 
what has been written on the importance of data and their use, which will 
allow an ever-growing audience to enrich their knowledge on the subject 
of education and be a means of giving life to always new ideas for further 
reflection. 
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1. Geographical differences 
in Italian students’ English test performance: 
a role of individual and local characteristics 


by Jana Kopecna, Cecilia Bagnarol, Silvia Donno, Michele Marsili 


Several studies have been focused on the analysis of the factors that affect the 
school performance. It has been found that social, economic and environmental 
factors have an important role in determining the educational achievement, and 
that individual characteristics and family background affect the school perfor- 
mance more than school characteristics. The spatial statistical analysis has been 
used by many researchers to study educational achievement. Principally, these 
studies concentrated their attention on the relationship between school perfor- 
mance and socio-economic variables of the catchment areas from the spatial 
perspective, by using Geographically Weighted Regression (GWR). 

The aim of this study is to investigate which individual characteristics may 
affect test scores in English Reading and Listening and which local character- 
istics of the catchment areas influence students’ achievement in these tasks. 

The data we used come from the National INVALSI Assessment of 8th 
grade for the 2018/2019 academic year and includes results of the standard- 
ized test in English Reading and Listening. 

Firstly, through the simple correlation analysis, we defined the individual 
and local characteristics that were then used as the explanatory variables in 
our analysis. Secondly, the simple Ordinary Least Squares (OLS) regression 
was used to analyze the relation of students’ performance in both tasks to 
previously selected characteristics. 

To verify a presence of positive spatial autocorrelation within the Italian 
territory, we calculated a widely used spatial statistic Global Moran’s I. It 
helped us to estimate the strength of spatial correlation, and to test also the 
significance of the spatial correlation. 

The positive result of Moran’s I endorsed our intention to include the ter- 
ritorial dimension into our study, and therefore, the GWR was used to model 
spatially varying relationships between the students’ achievement and the 
explanatory variables tested with the OLS regression. 
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However, the emphasis here is not so much to determine whether or not 
exist relationships between school performance and catchment area char- 
acteristics, but to determine if there are any interesting spatial variations in 
these relationships. 


Numerosi studi hanno rilevato che le caratteristiche individuali degli stu- 
denti, lo status socio-economico e il luogo di origine influenzano i risultati 
scolastici molto piu che le caratteristiche della scuola. Numerosi ricercatori 
hanno utilizzato l'analisi statistica spaziale per studiare gli apprendimenti 
scolastici. Tali studi hanno focalizzato l'attenzione sulla relazione esistente 
tra le performance scolastiche e le variabili socio-economiche del bacino di 
utenza da un punto di vista spaziale, facendo uso dei modelli di regressione 
Geographically Weighted (GWR). Le variabili piu frequentemente impiegate 
in tali modelli di regressione sono relative alle caratteristiche dello studente, 
del docente e della scuola, ad esempio la percentuale di famiglie con piu 
basso reddito, la percentuale di disoccupati, il titolo di studio dei genitori, 
il background familiare dei genitori (percentuale di famiglie monogenitore), 
l'ampiezza della classe degli studenti e la carriera dei docenti. 

L'obiettivo di questa ricerca é investigare quali caratteristiche individua- 
li possano avere effetto sui risultati delle prove INVALSI in Inglese Reading 
e Inglese Listening e quali caratteristiche locali del bacino di utenza della 
scuola influenzino gli apprendimenti scolastici in entrambe le materie. 

I dati utilizzati provengono dalle Rilevazioni nazionali INVALSI relative 
all'a.s. 2018/2019 per il grado 8 (III secondaria di I grado) per Inglese Re- 
ading e Listening. 

In primo luogo, attraverso un 'analisi della correlazione semplice, abbia- 
mo definito le caratteristiche individuali e locali da poter utilizzare come va- 
riabili esplicative nel modello di regressione spaziale. In seguito, un modello 
di regressione Ordinary Least Squares (OLS) e stato utilizzato per analizzare 
la relazione tra performance scolastiche in entrambe le materie di interesse 
e le suddette caratteristiche individuate. 

Per verificare la presenza di autocorrelazione spaziale globale positi- 
va sul territorio nazionale, é stato calcolato l'indicatore I di Moran, che 
consente inoltre di stimare la forza della correlazione spaziale e testarne la 
significativita. I risultati cosi ottenuti hanno confermato la nostra intenzione 
di includere nel nostro studio la dimensione territoriale e di conseguenza 
di utilizzare un modello di regressione GWR per studiare le variazioni ter- 
ritoriali nella relazione tra apprendimenti scolastici e variabili esplicative 
individuate e analizzate mediante la regressione OLS. 
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1. Introduction 


The issue of territorial inequalities has been widely discussed at a scientif- 
ic and institutional level: a substantial part of literature (Braga and Checchi, 
2010; Cipollone, Montanaro and Sestito, 2010; Montanaro, 2009; Falzetti, 
2019; Argentin et al., 2017) shows that the regional inequalities in economic 
growth and development along the North-South axis have had impact along 
the same axis even in a considerable discrepancy in efficiency-effectiveness 
of the school and education system. 

The ability of schools to ensure the same teaching-learning conditions 
for all students is an important indicator of the degree of equity of the Italian 
educational system. Besides educational uniformity in the effectiveness of 
individual schools (value added) also weigh different territorial distribution 
of Economic, Social and Cultural Status index (ESCS) and the different mi- 
gration background (INVALSI, 2010) of students and families. 

The data from the National Institute for the Evaluation of the Education 
and Training System (INVALSI) show that students attending schools in the 
North of Italy have a higher level of skills than those who attend similar 
schools in the South of the country (INVALSI Report, 2019). 

Since the 2017/2018 academic year, the students of the 5th of primary and 
3rd grade of secondary school have been subjected also to tests of English 
Listening and Reading comprehension. 

The students were evaluated in two ways: by assigning a score on the 
Rash scale and by assigning a level of proficiency in the language based on 
the standards defined by the Common European Framework of Reference for 
Languages (CEFR). 

The INVALSI Report 2019 confirmed that in both Listening and Reading 
tests, as it has happened already for Italian and Mathematics, the northern re- 
gions, with the exception of Piedmont, obtained a significantly higher score 
respect to the Italian average (200), while the South and Islands regions had 
significantly lower scores, with the exception of Abruzzo and Molise. 

Starting from these evidences, we analyzed the results of the INVALSI 
English tests of the 2018/2019 academic year of first grade secondary school 
students in all Italian regions. The aim was to identify the factors that, individ- 
ually or in interaction with each other, have a determining role in achieving 
adequate school skills or in defining disadvantaged situations in some geo- 
graphical areas. Initially, the geo-referencing of all Italian educational insti- 
tutions was conducted, and the spatial autocorrelation analysis was applied. 
Through an OLS regression model we analyzed the impact of the selected 
individual and territorial factors, and finally the GWR model allowed us to 
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evaluate the territorial differences in the relationship between these factors 
and student performance in English. 

Thus, in this study, we will briefly review the literature on the topic, de- 
scribe the INVALSI data used in the analysis, and rationalize decisions about 
different methodological approaches. Finally, we will present and compare 
the results obtained by applying selected methodologies on the data. 


2. Literature review 


Numerous researchers have focused their studies on the analysis of the 
factors that affect the school performance. Several studies have found that 
student individual characteristics and socio-economic variables and the lo- 
cation of origin affect student outcomes more than school characteristics 
(Taylor and Yu, 2009; Saifi and Mehmood, 2011). Various authors have used 
spatial statistical analysis to study educational achievement. Primarily, these 
studies focused on the relationship between school performance and so- 
cio-economic variables of the catchment area (Fotheringham, Brunsdon and 
Charlton, 2002; Gibbons, 2002; Gordon and Monastiriotis, 2007; Xiaomin 
and Shuosheng, 2011) from the spatial perspective, using Geographically 
Weighted Regression (GWR). The strong relationship between socio-eco- 
nomic status (SES) and school performance has been shown, pointing out 
that in the areas with worse socio-economic conditions, students still receive 
an inferior quality of education compared to their wealthier counterparts 
(Varadappa Naidoo et al., 2014). 

The most frequently analyzed variables are related to student, teacher and 
school characteristics: percentage of families with lower income, percentage 
of unemployment, parent educational level, parent family background (per- 
centage of families headed by a single parent), class size or how experienced 
are teachers. 


3. Data and methods 


The dataset used for the analysis was composed by the results of the CBT 
(Computer Based Test) survey in English Reading and Listening. The tests 
were carried out by the students attending third classes of the lower second- 
ary school (here after Grade 8) in the academic year 2018/2019. 

We decided to follow the macro-themes presented in the INVALSI re- 
ports (INVALSI, 2018, 2019), in particular gender gap, socio-economic 
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background and the importance of the origin of birth. We also included the 
regularity of studies as an indicator of academic performance. 

For a further analysis, the dataset variables were associated also with the 
information provided by the Student Questionnaire, containing data of the 
students and their family background (e.g. the number of books available 
at home, the possibility to study in a quiet environment, a computer to be 
used for the study with the relative software, etc.), used for the calculation of 
the indicator on the student’s socio-economic-cultural background (ESCS)! 
(Campodifiori et al., 2010). 


3.1. Notes on the geo-referencing of educational institutions 


The process of geo-referencing means to assign to each observation a 
specific geographic position expressed by the relative Cartesian coordinates, 
latitude and longitude. For the geo-referencing of the Italian school institu- 
tions, a dataset has been created containing the following variables: 

— school institution identification number; 
— address; 

— name ofthe city; 

— postal code (CAP). 

The process of converting the addresses into geographical coordinates 
was carried out through the Google Maps API (Application Programming 
Interface). The coordinates were obtained from the institutions addresses 
by querying the browser through the following URL address: https://maps. 
googleapis.com/maps/api/geocode/json?address=routet+street ^ number-ci- 
ty+postalcode&key=API key. In response to the request, the server returned 
the latitude and the longitude associated with addresses in JSON format. 

Using KNIME Analytics Platform (Berthold et al., 2008), an open-source 
data analytics, reporting, machine learning and data mining platform, the 
URL addresses were created by making use of the variables contained in the 
dataset (address, postcode, city). Subsequently, the software allowed us to 
make numerous "requests" simultaneously and thus we obtained as a “re- 
sponse", in JSON format, the coordinates for each school. Finally, the data 
in JSON format were converted into tabular format in order to prepare a new 
functional dataset for subsequent spatial statistical analyses. 


! For further details about ESCS calculation please refer to the following links: https:// 
www .istruzione.it/snv/allegati/01 A INVALSI escs slide.pdf and https://www.invalsiopen. 
it/indicatore-escs-valutazione-equa/. 


13 


ISBN 9788835131076 


3.2. Notes on the Univariate and Bivariate Global and Local Spatial 
Autocorrelation Indices 


Spatial autocorrelation (or spatial association) is a concept that derives 
from the observation that the values assumed by a variable are not distributed 
independently over the territory but, on the contrary, tend to concentrate in 
certain areas (Demarinis et al., 2011). In particular, we talk about: 

— positive spatial autocorrelation when similar values of a variable tend to 
group close to each other; 

— negative spatial autocorrelation when dissimilar values of a variable tend 
to group close to each other; 

— absence of spatial autocorrelation (or spatial independence) when the 
distribution of values over the territory is random. 

Generally, when applying spatial autocorrelation techniques, we distin- 
guish the methods for measuring on the entire set of localities (global meas- 
ures) and the methods for measuring on a spatially delimited subset of local- 
ities (local measures). 

The most frequently used measure to test the degree of global spatial 
autocorrelation is the *Moran's I" statistic (Moran, 1948), expressed by the 
formula: 


N GW X)(xj =) 


I=— 


Leb; Wy Lilo — e)? 


where: 

- Nis the number of observations (i.e. localities or geographical units); 

- x; 1s the value of the observed variable in locality i; 

- x; is the value of the observed variable in the locality j; 

- xis the sample mean; 

- (x;-x) represents the deviation from the average of the observed variable; 

- w,;is a weight assigned to the relationship between locality i and locality j. 
Moran's I statistic is structurally similar to the correlation coefficient and 

assumes values between -1 and +1. However, unlike the correlation coefficient, 

Moran’s I index does not assume a theoretical null value in relation to the inde- 

pendence condition, but rather a negative value very close to zero and equal to: 
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Therefore, to verify the presence of spatial autocorrelation it is necessary 
to compare values of I with the theoretical average E(/): 

— values of I greater than the theoretical average E(/) indicate positive spa- 
tial autocorrelation; 

— values of I lower than the theoretical average E(/) indicate negative spa- 
tial autocorrelation. 

The Moran’s I can be extended to the multivariate case: in this way the 
index represents the systematic association between the values of a variable 
observed x in a given area of interest and the values of another variable y 
observed in neighbouring areas. 

The bivariate Moran index of x with respect to y is thus obtained: 


N wvN 
i=1 Lj=1 Wij Zi Vj 


L PM Len! Acai: AGER, NN 
i CE Ej) 


Local measures, statistics for measuring the degree of spatial autocorre- 
lation at the local level, allow us to identify the contribution of each location 
on global pattern and therefore allow to study the variations of spatial au- 
tocorrelation within the territory. By focusing on each location, these tech- 
niques can be used to identify the presence of spatial clusters. 

Generally, a Local Indicator of Spatial Association (LISA) is any statis- 
tics characterized by the following features: 

— for each observation, the LISA provides a measure of the significant spa- 
tial concentration of similar values around the observation itself; 
— the sum of the LISA of all observations is proportional to a global spatial 

association indicator (Moran’s I). 

The most frequently used LISA index is represented by the local version 
of Moran’s I index and it is defined by the relationship: 


xj—X 


NL M Wi (x — X 
i dau EP ida ul ? ) 


To verify the presence of local spatial autocorrelation it is necessary to 
compare the values of /; with their theoretical average E(/,): 


Èj Wij 


E(1;) = N—1 
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In particular, positive values indicate the presence of clusters in which 
the observations have similar intensities (i.e. they all have high values or low 
values). On the contrary, negative values indicate the presence of clusters in 
which the observations have different intensities (1.e. the observations with 
high values are located close to observations with low intensity or vice ver- 
sa). Finally, when combining the significance information provided by the 
LISA in a map, we obtain the “Moran Significance Map” that reports the re- 
gions with significant LISAs associated with the relative positive or negative 
local spatial autocorrelation value (Anselin, 1995). 


3.3. Notes on the OLS and GWR spatial regression models 


The traditional OLS model is specified as a linear relationship between 
a dependent variable (y) and a set of explanatory variables (X) as follows: 


y = XB + e for e~N(0,07) 


where y is the dependent variable (in vector form, with N rows), X is a 
matrix with observations on K explanatory variables (with N rows and K 
columns), f is a vector with K regression coefficients (i.e., of dimension 
K by 1), e is a random error term (in vector form, with N rows), o? is the 
population error variance, and / is an identity matrix of dimension N by N 
(Chasco, 2013). 

The GWR is a non-stationary technique that models spatially varying 
relationships. Compared with a basic (global) regression, the coefficients 
in the GWR are functions of spatial location. Fotheringham, Charlton and 
Brunsdon (1998, 2002) give a general form of a basic GWR model as: 


m 


yi = Bio + 2. Pax T4 
pai 


Where y; is the dependent variable at location i; x, is the kth independent 
variable at location i; m is the number of independent variables; £,, is the 
intercept parameter at location i; f; is the local regression coefficient for the 
kth independent variable at location i; and e; is the random error at location i. 
The GWR allows coefficients to vary continuously over the study area, and 
a set of coefficients can be estimated at any location-typically on a grid so 
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that a coefficient surface can be visualized and interrogated for relationship 
heterogeneity. The GWR makes a point-wise calibration concerning a “bump 
of influence”: around each regression point where nearer observations have 
more influence in estimating the local set of coefficients than observations 
farther away (Fotheringham, Charlton and Brunsdon, 1998). 

With the GWR, each observation, or in our case each school, do not have 
the same weight as if all schools shared the same position (OLS approach); 
it is possible instead move over Italian territory and weight schools accord- 
ing to their proximity. So it is very crucial to choose the appropriate spatial 
weighting function or kernel of the GWR regression model. In heteroge- 
neous areas an adaptive kernel is the most appropriate, since it can select an 
optimal adaptive number of neighbours according to the density of the area. 


4. Results 


The smallest statistical unit we used in our study was the school. There- 
fore, the students’ individual scores were aggregated at the school level (for 
English Reading the average WLE for school is equal to 202.1 with SD equal 
to 16.5, while for English Listening the average WLE is equal to 199.3 with 
SD equal to 18.7). 

A data cleaning was carried out in order to exclude non-representative 
schools (i.e. schools with less than 20 students, average number of all class- 
es) and schools located on the little islands, since the distance from the main- 
land would affect the calculation of the Moran index. The study variables 
included: 

1) the standardized test score in the English Reading (WLE ERE) and En- 
glish Listening (WLE ELI) tests, expressed on a quantitative scale (Rash); 

2) the values of the ESCS socio-economic and cultural status index aggre- 
gated at the school level; 

3) the percentage of females calculated at the school level; 

4) the percentage of regular students in the matter of the course of study 
calculated at the school level; 

5) the percentage of native’ students calculated at school level. 


? Native students are born in Italy from parents born in Italy. 
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4.1. Spatial autocorrelation analysis: Moran’s I and LISA Cluster 
Map 


To understand better the distribution of observed variables we applied the 
spatial autocorrelation methods, specifically Moran’s I index, Moran’s Scat- 
terplot (Anselin, 1996) and Local Indicator of Spatial Association (LISA) 
(Anselin, 1995), using GeoDa, a software for analysis of spatial data (Anse- 
lin, 2003). 

Moran’s I provides an indication of the degree of linear association between 
the observed values of the study variable and the spatially delayed values. 

Spatial autocorrelation measures such as Moran's I require the construc- 
tion of the weight matrix that defines a neighbourhood for each geographical 
unit. The value of the study variable for each unit is compared with the aver- 
age weight of the values of the neighbouring units. 

To construct the matrix of weights, based on the spatial distances between 
two points expressed in terms of latitude and longitude, the “great circle dis- 
tance" or “arc-distance” were used, taking into account the terrain curvature. 
The average number of analyzed “neighbours” was 145, the median 83. 

The univariate Global Moran’s I revealed the presence of spatial auto- 
correlation in the distribution of the English Reading and Listening scores, 
respectively I=0.4531 and I=0.5799, and in both cases the index was signifi- 
cant (the pseudo p-value is «0.001). 

In order to investigate the factors that in some way affect the territori- 
al patterns identified by the univariate spatial analysis, the exploration was 
extended to the bivariate analysis, taking into account the surrounding so- 
cio-economic context. 

As previously mentioned, based on the assumption that in a given geo- 
graphical unit the values of the observed variable show a systematic asso- 
ciation with another observed variable in adjacent geographical units, the 
bivariate Moran’s I was used to explore and analyze the spatial dependence 
between the WLE score and the socio-economic and cultural index ESCS. 

The bivariate Global Moran’s I revealed the presence of spatial depend- 
ence between the distribution of the ESCS and the WLE scores of Reading 
and Listening, respectively I~ 0.2656 and I~ 0.2946, in both cases the index 
was significant (the pseudo p-value is <0.001). 

However, the Moran’s I index does not allow to verify whether spatial de- 
pendence generates clusters of schools by level of educational performance, 
nor to identify geographical boundaries of these clusters. In order to over- 
come these limits, the Moran’s Scatterplot and local autocorrelation meas- 
ures were applied. 
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The Moran’s Scatterplot displays the values of the observed variable 
standardized on a Cartesian graph, expressed in standard deviation units (the 
average is set equal to 0 and the standard deviation equal to 1) on the x-axis 
and on the y-axis the corresponding spatially delayed values are also stand- 
ardized. The index I represents the angular coefficient of the linear relation- 
ship between the two variables reported on the axes: if the points represented 
are dispersed between the four quadrants this would indicate the absence of 
a correlation (the angular coefficient is zero). If, instead, a correlation exists, 
the scatterplot would allow to distinguish the different types of spatial corre- 
lation (High-High, Low-Low, High-Low, Low-High). 

Subsequently, the results of the scatterplots were displayed in the map 
in order to depict the areas with different types of correlation: this type of 
representation made it possible to verify the geographic proximity of the 
schools of the same type of correlation and to identify some homogeneous 
clusters. 

Nevertheless, the Moran’s Scatterplot does not provide information on 
the significance of the identified spatial clusters. Thus, the Local Indicator 
of Spatial Association (LISA) was calculated for each school, in order to 
measure the interdependence with other schools and to indicate the type of 
the spatial relationship (positive or negative) and its significance. 

The results of the bivariate LISA indicator were displayed in the map in Fig. 
1 and 2, in which 4 different clusters of educational institutions were identified: 
- the first High-High cluster, is composed of schools that achieve WLE 

English scores above the national average and are surrounded by institu- 

tions whose ESCS is above the average; for both tasks, all regions of the 

Centre-North of Italy are involved, except for the region of Valle d’ Aosta; 
- the schools that form the second, Low-High cluster, are schools with low 

WLE scores that do not seem to benefit from their geographical location 

in areas where the surrounding schools have on average higher values of 

the ESCS. These schools are present in the Centre-North, especially in 
eastern regions like Piedmont, Lazio and Abruzzo; 

- the third Low-Low cluster, is made up of schools that register low WLE 
scores and are adjacent to schools with the ESCS lower than the national 
average; primarily it includes the regions of Southern Italy and the Islands; 

- the last High-Low cluster, is formed by schools with WLE scores above 
the national average that are surrounded by schools with the ESCS below 
the national average. The schools belonging to this cluster, we can define 
the resilient ones, are present mainly in the regions of the South and Is- 
lands, but also in some contexts of the Val d'Aosta, Trentino Alto Adige 
and the provinces of lower Lazio. 
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Fig. 2 — Bivariate LISA cluster map, WLE Listening and ESCS 
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4.2. OLS and GWR Regression 


Our analysis starts with a global approach: the data are used to compute 
a single statistic for each parameter for the entire observed area and the rela- 
tionship between variables are stationary across the study area. 

While, in the traditional OLS all places have the same weight as if all 
places shared the same location, in the GWR, as we move over space, obser- 
vations are weighted according to their proximity to a location. The weighted 
calibration implies that the weighting of an observation is not constant but 
varies with 1. In these cases, we speak about the local and global statistics. 

Multivariate linear regression was performed on the data using OLS mod- 
el. Subsequently, the GWR model was applied to deal with spatial non-sta- 
tionarity (Varaidappa Naidoo et al., 2014). For both global and local regres- 
sion, the dependent variable was the students’ score in English (Listening 
and Reading) tests. The four independent variables used in the study were 
the ESCS index, percentage of female, regular and native students calculated 
for each school. The analyses were performed using statistical software R 
v3.4.4, using the “spgwr” package (Bivand and Yu, 2012) and the results 
were imported into Tableau for spatial mapping. 

The variables were standardized in order to make transversal comparisons, 
therefore, the unitary impact can be evaluated in terms of standard deviation. 
With all conditions being equal (average values of predictors), better perfor- 
mances are recorded in Reading than in Listening. The ESCS index has a great 
weight on both tasks, but greater on Listening: it is possible to hypothesize 
that more economic possibilities provide more tools (during their out-of-school 
time) to children to improve their learning such as speaking with native speak- 
ers, attending private English lessons or participating in a language course 
abroad. All these experiences have a substantial impact on performance in 
English Listening. As emerged from the national INVALSI Report (INVALSI, 
2019), girls have better results in Listening test, thus schools with a higher 
percentage of girls have better results. As the percentage of natives increases, 
the school performance worsens: even the results of the INVALSI Report 2018 
(INVALSI, 2018), infact, show that foreign children have a greater propensity 
to learn other languages. As the percentage of regulars or anticipators grows, 
performance improves especially in Reading. 

All the coefficients are statistically significant and the fit of the OLS mod- 
el is good, Adjusted R-square is equal to 0.49 for Listening and 0.48 for 
Reading (Tab. 1). 


21 


ISBN 9788835131076 


Tab. 1 — OLS Regression Statistics 


Listening Reading 
Estimate Std. Error Estimate Std. Error 
Intercept 199:29%** 0.17 202.08*** 0.15 
ESCS 12.59*** 0.19 10.85*** 0.17 
Female (96) ].16*** 0.17 0.98*** 0.15 
Native (%) -3.96*** 0.20 -2.53*** 0.18 
Regular (96) 0.98*** 0.22 1.72*** 0.20 
= 0.49 = 0.48 
Adjusted = 0.49 Adjusted = 0.48 


*** p-value «0,001. 


In the correct OLS model residuals should not be clustered in location, 
however in our analysis the residuals were not spatially random (Fig. 3): the 
models tended to underestimate the North and overestimate the South. 


Fig. 3 — OLS Residuals distribution, Listening and Reading 


We applied a GWR regression model using adaptive kernel with varying 
neighbourhood size: this GWR model was able to explain about 81% of the 
variation for Listening and 73% for Reading. The output from the GWR mod- 
el revealed how the coefficients vary across Italy: the global coefficients are 
exactly the same as the coefficients in the previous OLS model (Tab. 2 and 3). 
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Tab. 2 — GWR parameter summary results, Listening 


Listening Min 1°Q Median 3^Q Max Global 
Intercept 176.28 189.17 202.22 209.44 217.94 199.29 
ESCS 1.75 6.98 8.54 10.04 14.46 12.59 
Female (%) -2.99 0.15 0.77 1.37 3.85 1.16 
Native (96) -8.83 -1.16 -0.38 0.50 6.38 -3.96 
Regular -2.79 1.02 2.11 3.13 7.98 0.98 
Quasi-global = 0.81 

Tab. 3 — GWR parameter summary results, Reading 

Reading Min 1°Q Median 3°O Max Global 
Intercept 182.52 195.77 203.99 209.76 215.35 202.08 
ESCS 3.54 6.44 7.74 9.05 13.26 10.85 
Female (%) -1.87 0.29 0.73 1.16 2.80 0.98 
Native (%) -3.40 -0.58 0.16 0.88 5.37 -2.53 
Regular -1.59 1.58 2.54 3.36 8.59 1.72 


Quasi-global — 0.81 


Fig. 4 — GWR Beta Estimated Coefficient — Intercept, Listening and Reading 


The values of the GWR intercept show how substantially Italy has a sad 


score gradient from the North (schools in darker grey) to the South (schools 
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in light grey). In the South we report a cluster of schools in Bari whose re- 
sults are in line with the national average (Fig. 4). 

High parameter estimates mean that the effect of the variables is higher in 
particular areas as compared to others. 

From this point forward, we have decided to visualize in the map only 
those schools that had a statistically significant coefficient estimated. For 
the ESCS index, almost all the schools of the dataset (more than 99%) are 
statistically significant. The estimated coefficients are all positive as proof of 
the fact that the direction of the influence of this indicator, across the national 
territory, is the same. It is possible to identify important spatial clusters, the 
indicator has a greater impact on dark grey schools that are mainly located 
in the Centre-South of Italy with the exception of a range of schools that are 
from Foggia-Potenza-Matera-Salerno, of Sardinia and the western provinces 
of Sicily. In northern Italy we can find clusters of schools in which the indi- 
cator had greater weight in Milan and Turin in Listening when compared to 
Reading (Fig. 5). 


Fig. 5 — GWR Beta Estimated Coefficient. ESCS index, Listening and Reading 


The map in Fig. 6 shows the statistically significant estimated coefficients 
for the variable percentage native which correspond to about 20% of the 
schools in Listening and 10% in Reading of the dataset. Also in this case we 
see that there are spatial clusters and a North-South gradient, it is possible to 
assume that the variable has a negative effect in the North where there are 
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schools with a higher percentage of foreigners. We also reported a small group 
of schools in Lecce that had similar values to the Verona schools in Listening. 
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Fig. 7 — GWR Beta Estimated Coefficient. Females (%), Listening and Reading 
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Fig. 6 — GWR Beta Estimated Coefficient. Regular (%), Listening and Reading 


Maps in Fig. 7 and 8 report respectively the statistically significant esti- 
mated coefficients for the variable percentage of female and regular: maps 
show that there is a general positive impact throughout the national territory 
except in some schools in Lecce and Naples in the first case and in the prov- 
ince of Trento in the second one. 


5. Conclusion 


Nowadays, Italy still shows substantial differences in scholastic achieve- 
ment. As a matter of fact, differences in skills and learnings are one of the 
main foundations of socio-economic development among territories. 

In this perspective, it is crucial to perform analyses that include territorial 
complexity. Moreover, the schools that are closer have a bigger influence 
than those distant ones, referring to the “First Law of Geography” enunciated 
by Tobler in 1970: “Everything is related to everything, but near things are 
more related than distant things”. 

Therefore, the local approach is a useful tool to study territorial differenc- 
es in the educational field, since spatial thinking will have an increasingly 
important role to play. 

Starting from geo-referencing of schools in the Italian territory, it was 
possible to reveal a heterogeneous structure in the geography of learning. All 


26 


ISBN 9788835131076 


the analyses carried out showed that this spatial heterogeneity identifies new 
and changing territorial continuums. 

The aim of this research was to study in detail this heterogeneity by using 
data at the student level and subsequently aggregated at the school level. 
Indeed, we demonstrated, in conformity with the evidence in the literature, 
that students’ individual characteristics play a greater role than school char- 
acteristics. 

The results of univariate and bivariate spatial autocorrelation analyses 
showed that there are spatial clusters in Italian territory that reveal a sad 
score gradient in educational performance from the North to the South. 

However, going deeply, beyond the administrative boundaries, it is possible 
to find, for example, schools in high performing clusters (located in the North 
of Italy) that perform worse respect to the other schools in the same cluster, and 
vice versa. These observations constitute clusters of spatial outliers. 

Given these premises, the use of a local approach, in addition to the glob- 
al one, is the best option because it allows us to better understand the phe- 
nomenon. In fact, the geographically weighted model provided the best fit to 
the data. While in the traditional OLS model a global parameter is estimated, 
with the GWR approach we obtain single local parameters. Thus, instead 
of calibrating a single regression equation, the GWR generates a separate 
regression equation for each observation (school) contained in the dataset. 
As we move over space, schools are weighted according to their location in 
the territory. 

We found spatial heterogeneity in the effects of the predictors on English 
performance, especially related to the ESCS index. The examination of geo- 
graphical variations could help us to better understand the uncover relevant 
variables for improving model performance and better understand the Italian 
complexity in order to identify new spatial patterns. The identification of 
these spatial clusters and also their important exceptions can provide new 
tools to manage new future challenges in this complex field. 
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2. How important is the time spent 
with children on their performance?" 


by Valeria F. Tortora, Patrizia Giannantoni, Paola Giangiacomo 


A significant advantage at school and in life is certainly represented by 
the time spent by parents with their children, especially in their schooling. 
Being able to motivate and stimulate their children, nurture a real interest 
and actively engage are a remarkable added value for improving school per- 
formance. 

Helping to undertake school challenges together does not necessarily 
mean helping children do their homework, but, as demonstrated by an anal- 
ysis of the OECD PISA 2009 data on the data collected through the family 
questionnaire, there is no need for a PhD title or a very large amount of time 
for parents to make a difference. 

In the international survey IEA PIRLS 2016 all parents in the sample 
were asked about extracurricular activities (such as reading books, telling 
stories or playing with forms) carried out together with their parents. The 
main purpose of this work is to develop a reflection, supported by empirical 
evidence, on the existence of a close relationship between the involvement 
of parents with children in preparatory activities for learning and the attitude 
of the latter towards reading, as well as their performance in reading and in 
Italian in class II and V of elementary school (grade 4 and 5). 

Grade 4 performance is obtained from IEA PIRLS 2016 surveys, while 
grade 2 Reading and grade 5 for Italian from respectively the 2013/2014 
and 2016/2017 INVALSI Survey; therefore, the study uses the longitudinal 
database obtained by merging IEA PILRS 2016 and INVALSI tests through 
longitudinal data match. Students who have taken all the tests are the refer- 


* This chapter is the output of a joint work of the three authors. Nonetheless, V. F. Tortora 
and P. Giangiacomo contribute substantially to section 1 “Introduction”, section 2 “Objective 
of the research" and section 5 “Discussion”, P. Giannantoni contributes substantially to sec- 
tion 3 *Data and Methods" and section 4 "Results". 
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ence population. The spread and characteristics of parental involvement with 
children will be outlined with descriptive analysis in the first part of this con- 
tribution. In a second part, the connection between parental involvement and 
academic performance will be deepened by regression models, controlling 
for the student’s gender, educational level of the family, previous attendance 
of kindergarten. 


Un notevole vantaggio a scuola e nella vita é sicuramente rappresentato 
dal tempo trascorso da parte dei genitori con i propri figli, in particolar 
modo nel loro percorso scolastico. Riuscire a motivare e stimolare i pro- 
pri figli, nutrire un vero interesse e impegnarsi attivamente rappresentano 
un valore aggiunto inestimabile per migliorare le performance scolastiche. 
Contribuire a intraprendere insieme le sfide scolastiche non significa ne- 
cessariamente aiutare i propri figli nel fare i compiti a casa ma, come si é 
dimostrato attraverso un 'analisi dei dati dell'OCSE PISA 2009 sui dati rac- 
colti tramite il questionario famiglia, non serve un dottorato di ricerca o un 
numero illimitato di ore per far si che i genitori facciano la differenza. Nella 
rilevazione internazionale IEA PIRLS 2016 venivano rivolte ai genitori de- 
gli studenti partecipanti, attraverso la somministrazione di un questionario 
famiglia, alcune domande concernenti le attività extrascolastiche svolte in- 
sieme ai figli, come ad esempio leggere libri, raccontare storie o giocare con 
le forme. La principale finalità di questo lavoro è quella di sviluppare una 
riflessione, sostenuta dall'evidenza empirica, sulla esistenza di una stretta 
relazione fra il coinvolgimento dei genitori con i figli in attività propedeu- 
tiche all'apprendimento e l'atteggiamento di questi ultimi verso la lettura, 
oltre che la loro performance in Lettura e in Italiano nelle classi IV e V della 
scuola elementare. La performance in classe IV è ottenuta dalle indagini IEA 
PIRLS 2016, mentre in classe V dalla Rilevazione INVALSI 2016/2017; dun- 
que lo studio utilizza come base dati quella della IEA PILRS 2016 e quella 
INVALSI 2016/2017, evidenziando solamente gli studenti che avranno svolto 
entrambe le prove. La diffusione e le caratteristiche del coinvolgimento ge- 
nitoriale con i figli verranno delineati in una analisi descrittiva nella prima 
parte di questo contributo. In una seconda parte il legame che intercorre tra 
coinvolgimento genitoriale e rendimento scolastico verrà approfondito tra- 
mite modelli di regressione, controllando per genere e origine dello studente, 
livello socio-economico della famiglia, frequenza pregressa di asilo nido e 
scuola dell'infanzia. 
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1. Introduction 


The most of studies have signaled the beneficial effects of parental in- 
volvement in children’s educational lives and have examined the possible 
impact of parental characteristics, such as family income and parental edu- 
cation, on children’s educational outcomes (Castro et al., 2015). 

This paper evaluates the levels of parental involvement as well as the 
relationship of involvement with cognitive (reading performance) outcomes. 

Studies have suggested that parental involvement in children’s activities 
may signal the route through which a parent’s skills and motivations are 
transferred to children and should be positively associated with children’s 
cognitive and other development (Baker and Scher, 2002). 

Several studies have also noted the differential effects of parental expec- 
tations and parenting behaviours on children’s education. They indicated that 
parental expectations and attitudes, rather than specific behaviours such as 
involvement in school activities, better explained children’s academic out- 
comes (Fan and Chen, 2001; Reynolds and Gill, 1994). 

The idea that parental involvement has positive influence on students’ 
academic achievement is so intuitively appealing that society in general, and 
educators in particular, have considered parental involvement an important 
ingredient for the remedy for many problems in education. 

Society in general, and educational researchers in particular, have long 
been interested in the positive effect that parental involvement may have on 
students’ academic achievement (OECD PISA, 2018). The perception that 
parental involvement has positive effect on students’ academic achievement 
is so intuitively appealing that policy makers, school board administrators, 
teachers, parents and even students themselves, have agreed that parental 
involvement is critical for children’s’ academic success. 

Although parental involvement is often simplistically perceived as unidi- 
mensional, in reality, it is probably better to conceptualize this construct as 
being multifaceted in nature, because parental involvement subsumes a wide 
variety of parental behavioral patterns and parenting practices (Hill and Tay- 
lor, 2004). Such an approach has been adopted in several recent empirical 
studies (Singh e? a/., 1995). Furthermore, there is evidence indicating that 
certain dimensions of parental involvement may have more noticeable effect 
than some other dimensions on students’ academic achievement (Singh et 
al., 1995). 

Findings suggest that some forms of parental involvement are more 
strongly related to cognitive outcomes than others. These include reading to 
children when they are young (OECD PISA, 2018). 
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Students whose parents often read books with them during their first year 
of primary school show markedly higher scores in PIRLS 2016 than students 
whose parents read with them infrequently or not at all (Mullis et al., 2017). 

The performance advantage among students whose parents read to them 
in their early school years is evident regardless of the family’s socio-eco- 
nomic background. Parents’ engagement with their children is strongly asso- 
ciated with better performance (Mullis et al., 2017). 

Most parents know, instinctively, that spending more time with their chil- 
dren and being actively involved in their education will give their children 
a good head-start in life. But as many parents have to juggle competing de- 
mands at work and at home, there never seems to be enough time. Often, too, 
parents are reluctant to offer to help their children with school work because 
they feel they lack some of the skills that would make a difference to their 
children’s success in school. The good news coming from analyses of PIRLS 
data is that it does not require a PhD or unlimited hours for parents to make a 
difference. In fact, many parent child activities that are associated with better 
reading performance among students involve relatively little time and no 
specialized knowledge. What these activities do demand, though, is genuine 
interest and active engagement. 

The parents’ responses show a close relationship between their own in- 
volvement with their child and their child’s engagement in reading-related 
activities during the first year of primary school. Students whose parents re- 
ported that they had read a book with their child “every day or almost every 
day" or “once or twice a week" during the first year of primary school have 
markedly higher scores in PIRLS 2016 than students whose parents reported 
that they had read a book with their child “never or almost never” or only 
“once or twice a month” (Mullis et al., 2017). 


2. The reasons for a research on parental participation and some 
studies 


School and family represent two educational systems within which the 
child grows and forms, two systems that have always interacted even if with 
sometimes different purposes, times and methods. As explained in the “In- 
troduction” paragraph, the correlation between parental involvement and 
student performance is significant, but despite this, in the literature this rela- 
tionship has been addressed in international surveys, promoted by IEA and 
OECD, only through the administration of some questions on questionnaires 
for students and families. 
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There are no large-scale surveys that photograph the effects of parents’ 
active participation in their children’s school life or on their academic per- 
formance. There are several local initiatives, promoted by both schools and 
universities, which investigate whether, and to what extent, parents partici- 
pate in school life and which strategies can improve this relationship. 

Some studies were carried out using data from the OECD PISA inter- 
national survey. Among the most relevant a mention is for the study car- 
ried out by Borgonovi and Montt, they studied parental involvement using 
PISA 2009 data, analyzing the relationship between parental involvement in 
school, parental involvement at home, early parental involvement in school 
life and children’s reading literacy, among other results (Borgonovi and 
Montt, 2012). 

Analyzing the data of 14 countries participating in the administration of 
the family questionnaire, they identified 3 types of involvement: academi- 
cally oriented involvement, non-academically oriented involvement and par- 
ents’ reading habits. The authors have underlined the importance of early in- 
volvement of parents to skills; some forms of home-based involvement, both 
academic and non-academic; and positive parents’ reading habits and atti- 
tudes showed a positive relationship with reading performance for 15-year- 
old children, even after controlling for the ESCS of the family. 

Again with PISA data but in the 2015 edition some Italian authors eval- 
uated the relationship between parental involvement and student science 
achievement (Caponera et al., 2019). The Italian data were compared with 
data from EU countries in the Mediterranean area which are culturally sim- 
ilar to Italy and which participated in the optional family questionnaire, 
namely France, Malta, Portugal and Spain. 

The results showed that ESCS contributed to the prediction of science 
achievement; furthermore, the results evidenced the positive, significant ef- 
fects of parental involvement, factors mediating the relationship between 
ESCS and PISA test achievement, especially in France, Malta and Portugal 
(Caponera et al., 2019). 

There are no studies on Italian data of the international PIRLS survey. 
Therefore, aware from the literature that in the fourth year of schooling pre- 
school activities are predictors of understanding the language, and driven by 
a strong interest in understanding to what extent parental support has effect 
already at the very beginning of the academic career, we projected this re- 
search, used for the first time data from PIRLS matched with the National 
Survey promoted by INVALSI. 
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3. Objective of the research 


The aim of the study is to investigate the relationship between involve- 
ment of parents in preparatory activities for learning with their children and 
the attitude and performance of the latter in language subjects. Particularly 
we looked at children’s attitudes towards reading, self-confidence when fac- 
ing test, as well as performance in Reading and in Italian in class II and V of 
primary school (grade 2 and 5), making use of a joint database constructed 
by the longitudinal match of PIRLS 2016 data on Italian students attending 
grade 4 of primary school and INVALSI data of grade 2 (a.y. 2013/2014) and 
grade 5 (a.y. 2016/2017). 


4. Data and methods 


In order to investigate the topic of our research, we built up an “ad hoc” 
dataset through a linkage between different surveys: International data from 
PIRLS 2016 Survey (students in the fourth year of primary school — grade 4) 
and INVALSI data from census survey of year 2013/2014 (students in second 
year of primary school — grade 2) and year 2015/2016 (students attending the 
fifth year of primary school — grade 5). More specifically, using the “SIDI 
INVALSI” code as linkage key it was possible to match information on the 
same student stemming from different surveys (national and international) 
and different academic years (2013/2014, 2015/2016, 2016/2017). 

Students in all the three surveys constituted the reference population for 
this research (see Fig. 1). 


GRADE 5 


INVALSI data 
Sidi INVALSI 2016/2017 


Pd 


GRADE 4 
u IEA PIRLS 
Sidi INVALSI 2015/2016 
GRADE 2 
INVALSI data 
2013/2014 


Fig. 1 — Longitudinal scheme of the final sample 
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The dataset created collected from PIRLS: 


— information about frequency of early parental literacy activities before 
school, the key explanatory variable of our analysis; 
— socio demographic information, (e.g. gender, education of parents, atten- 


ding kindergarten), used as control variables. 


From INVALSI tests we derived standardized scores in Reading and Lit- 


eracy come from INVALSI grade 2 and grade 5 tests. 


Furthermore, we included in the analysis specific indexes about attitude 
toward reading (i.e. pleasure for reading and confidence in reading) from 
PIRLS, together with Question 11 of INVALSI student Questionnaire which 
asks: “How much did you feel calm before the test?”. Standardized scores 
together with attitudes measurements represent the outcome variables of all 


the work presented in this paper. 


Table 1 illustrates the variables used, their characteristics and the survey 


from which they are originated. 


Tab. 1 — Scheme of variables included in the analysis 


Name Label Typol. Categories Source 

ELA-Scale Early Literacy Actitivites Scale Quant PIRLS G4 

ELA-Index Early Literacy Actitivites Index Categ Often PIRLS G4 
Sometimes 
Never 

Like Read-Index Student Like Reading Index Categ Often PIRLS G4 
Sometimes 
Never 

Conf Read-Index Student Confidence Reading Index Categ Often PIRLS G4 
Sometimes 
Never 

Feel Calm Test Feeling Calm during Italian Test Categ Notatall INVALSI G5 
A little 
Enough 
Very much 

Gender Gender of the student Categ Girl PIRLS G4 
Boy 

Parent Univ Parental University degree Categ No PIRLS G4 
Yes 

Preschool 2+ Preschool at least 2 years Categ No PIRLS G4 
Yes 

Reading score G2 INVALSI Reading score grade2 Quant — INVALSI G2 

Italian score G5 INVALSI Italian score grade 5 Quant — INVALSI G5 
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In order to give a more specific frame to the analysis, we decided to look 
to the relationship between pre-school activities only related to literacy and 
reading/language development, therefore we selected from the list of PIRLS 
item concerning parent-child activities only those related to language devel- 
opment and excluded those more related to numeric skills. Final items from 
PIRLS 2016 included in the analysis were nine (see Fig. 2), with 3-points 


29 6 


scale answer: “often”, “sometimes” and “never or almost never”. 


Before your child began primary/elementary school, how often did you or someone else in your 
home do the following activities with him or her? 
Often Sometimes Neveror 
almost never 
v v v 
1) Read books AAA Ae ee e eee O — © — © 
2) Tell stories -—————————————————————————— Q-——Q-——O 
3) Sing songs -—-——————————————7————————— Q——Q——QOG6 
4) Play with alphabet toys (e.g., blocks with letters of the alphabet) ---— Q —— Q = © 
5) Talk about things you had done -—__ O O) (O) 
6) Talk about what you had read ———7-—-————————————————— (e (n (C) 
7) Play word games -——————-————————————————— Q——O0-——O 
8) Write letters or words —-——————————————————————— —= O == (0) 
9) Read aloud signs and labels --—__ —O0—O—O 
«——————— 
Never 
Often | Sometimes | or Almost 
Never 
107 62 


Fig. 2 — Items composing Early Literacy Activities before beginning primary school 
Source: IEA PIRLS 2016 Methods and Procedure 


PIRLS Survey scored students were scored according to frequency of being 
exposed to the nine activities with their parents, using Item Response Theory, 
resulting in an Early Literacy Activities Scale (ELA-Scale). From this scale 
an Early Literacy Activities Index (ELA-Index) is derived, whose categories 
are: often, sometimes and never or almost never. The category "often" in the 
ELA-Index corresponds to scores higher than 10.7 in the ELA-Scale and rep- 
resents a situation where students were “often” engaged in at least five of the 
nine activities with their parents and “sometimes” engaged in the remaining 
four. On the opposite, the category never/almost never" in the ELA-Index col- 
lects students with scores lower than 6.2 in the ELA-Scale, which corresponds 
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to all those situations where five or even more activities have never been per- 
formed by parents with their children, before beginning primary school. 

A first part of analysis was directed to provide a description of frequency 
and typology of early literacy activities performed by Italian parents before 
school age of their children, highlighting those aspects that are more fre- 
quently practiced among parent-child activities. For all descriptive analyses 
sampling weights produced by PIRLS Survey were used. 

A second step of analysis was devoted to gain a first overview of the 
association between early literacy activities and attitude towards reading 
and standardized tests during primary school. We made use of PIRLS Early 
activities Index to classify children in those who had been exposed to par- 
ent-child activities before school age “often”, “sometimes” or “never/almost 
never" and used the PIRLS Index of “pleasure for reading” and “confidence 
in reading”, together with INVALSI Question from student questionnaire 
“How much did you feel calm before the test?". Association significance was 
tested through chi-squared test. 

A final section of the work is directed to explore the effect of early activ- 
ities on school performances, first by looking at mean differences between 
those exposed and not to ELA activities, then controlling for possible con- 
founder factors, in regression models. We decided to use INVALSI test score 
as a measure of abilities. In grade 2 INVALSI score is a quantitative variable 
on a scale from 0 to 100 (percentage of correct answers); in grade 5 we made 
use of WLE scores, which is a score estimated based on Rasch model, cen- 
tered on an average value of 200 and a standard deviation of 40. Significance 
of mean differences in scores was tested through ANOVA test. Since both the 
outcome variables (Reading and Literacy) were quantitative we opted for a 
linear regression. 


5. Results 
5.1. Frequency of occurrence of items of Early Literacy Activities 


Starting from the whole sample of 3063 students in PIRLS 2016 — grade 
4, we performed a first match retrospectively with INVALSI Reading test at 
grade 2 (2013/2014) and a second match forward with INVALSI Italian test 
at grade 5 (2016/2017), using as key variable a longitudinal code for students 
(“SIDI INVALST’). 

Students who took part of all the tests were the reference population. 
Furthermore, we excluded from the analysis those students for whom Italian 
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was not the first language, and all the students with missing values in any of 
the variable in the analysis, resulting in a final not-weighted sample of 2,260 
students. 

Based on this sample, we looked at the exposure to Literacy activities 
students had before beginning primary school, according to the nine items 
composing the PIRLS ELA-Index (Fig. 3). We can clearly observe as all 
the activities are frequently performed with a specific prevalence for “talk 
of what you have done", followed by “tell stories" and “sing songs", which 
can easily be seen as a part of the daily routine in Italian families. At the 
opposite, the least performed are activities such as “Book discussion” and 
“play word games” which are probably more challenging for children at a 
pre-school age. 


90,0 


40,0 B 

30,0 

20,0 

10,0 | | 
oo NE Nm (| | L I I I 


Read books Tell stories Sing song Ply Talk done Book Play word Write letter Read aloud Counting 
alphabet discussion games words signs words 


OOften Sometimes B Never or almost never 


Fig. 3 — Frequency of different items of parental early activities with children 


5.2. Association of ELA with attitudes towards reading and test 
anxiety 


Before looking to the effects of parental Early Literacy Activites on skills, 
we decided to explore the association of these same pre-school activities with 
attitudes towards reading and self-confidence of students in front of the test. 
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Table 2 illustrates a gradient (significant with chi-squared test) in the pro- 
portion of kids who “very much like reading” moving from students who had 
experienced frequent early literacy activities (44% like very much reading) 
to those who had never been engaged in this kind of activities in their fam- 
ilies (21% like very much reading). On the opposite, the dislike for reading 
among these 10-year students is about one over five (19%) in the group never 
engaged in Early Literacy, 6 percentage points higher than among those ex- 
posed to parent-child literacy experiences (13%). 


Tab. 2 — Association between parental early literacy activities and “Student like 
reading " index (%) 


Students like reading — Index 


ELA-Index Very much Somewhat Do not Total 
like reading like reading like reading 

Often 44 43 13 100 

Sometimes 38 45 17 100 

Never/almost 21 60 19 100 


An even sharper polarization can be observed for “confidence in reading" 
(Tab. 3). More than a half (55%) of students exposed to ELA in pre-school 
time feel very confident in reading at time of PIRLS Survey; whereas less 
than one over four (21%) among those not exposed to ELA have strong con- 
fidence in reading in grade 4 of primary school. The central category is quite 
stable, whereas the proportion of students not confident in reading is four 
times higher (42% against 9%) among those never engaged in pre-school 
literacy activities with respect to those often engaged. 


Tab. 3 — Association between Parental early literacy activities and "Student confi- 
dent in reading" index 


Students confident in reading — Index 


ELA-Index Very Somewhat Not confident Total 
confident confident 

Often 55 36 9 100 

Sometimes 44 40 16 100 

Never or almost never 21 37 42 100 


The last association observed was about the ability to control anxiety in 
front of standardized test, reacting with calm and self-confidence. This abil- 
ity seems more frequent among students who benefited of parental shared 
activities, compared to those who did not. In fact, although in Table 4 we can 
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see a quite similar distribution of students with different exposition to ELA in 
the extreme categories, looking at the intermediate category (“a little” calm 
and “enough” calm), we can detect a very uneven distribution in favor of 
those with parental activities background. 


Tab. 4 — Association between parental early literacy activities and feeling calm dur- 
ing the INVALSI test (?6) 


Feeling calm during the test 


ELA-Index Not at all A little Enough Very much Total 
Often 12 26 37 25 100 
Sometimes 12 26 39 23 100 
Never or almost never 14 71 0 15 100 


Note: The value of zero in one category is due to the small number of students in the category 
of never or almost never in the early literacy activity index in the original sample population. 


5.3. Reading and Italian performance according to ELA exposure 


The average scores at INVALSI test for Reading and Italian has a sharp 
decline from students with an experience of early parent-child activity and 
those who did not (Tab. 5). 

Percentage of correct answers in Reading declines from 89.2 to 81. 

We should consider that the distribution of Reading percentage scores 
is highly asymmetric, with over 70% of students with a score higher than 
85, very close to the top performance (which is obviously 100). Thus, the 
gap between students exposed and not exposed to parental activities appears 
extremely meaningful, as confirmed by Anova test, significant at 0.05 level. 


Tab. 5 — Mean scores at INVALSI Reading Test (grade 2) according to different cat- 
egories of ELA-Index 


INVALSI Reading score(% ) — Grade 2 


EARLY Lit Act Index Mean N Std Dev 
Often 89.2 157,816 15.8 
Sometimes 87.1 139,419 16.4 
Never or almost never 81.0 1,866 16.6 
Total 88.2 299,102 16.2 


Notes: Anova test is significant at level 0.05; number of students is weighted. 
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Looking at Table 6 we can see that WLE scores in Italian have a 20 points 
difference (significant at the Anova test with a level of 0.05) between the 
two extreme categories “Often” and “Never/Almost Never” ELA activities, 
which corresponds to 0.5 standard deviation in the WLE scale. It is interest- 
ing to note that the overall mean scores in the sample is quite high (207.1), 
but kids never engaged in Early literacy activities before school significantly 
underperform with respect to this average value. 


Tab. 6 — Mean scores at INVALSI Italian Test (grade 5) according to categories of 
ELA- Index 


INVALSI Literacy score (WLE) — Grade 5 


Early Lit Act Index Mean N Std Dev 
Often 209.3 157,816 36.5 
Sometimes 204.9 139,419 37.9 
Never or almost never 188.0 1,866 28.7 
Total 207.1 299,102 37.2 


Note: Anova test is significant at level 0.05, number of students is weighted. 


Aware that these differences can be driven by many other factors linked 
together with parent-child preschool activities, we tried to correct for this 
potential bias examining the effects of Early Literacy Activity in a Linear re- 
gression model together with control variables: gender, frequency of 2 years 
or more of pre-school, university degree of at least one parent. The Early Lit- 
eracy activity is here measured by the ELA-Scales, as quantitative variables, 
in order to encompass the shortcoming of very small number of students in 
the category of those who experience "never or almost never". 

The first model (Tab. 7) is built to predict Reading scores at INVALSI 
test of grade 2 based on both characteristics of the student and his/her family 
(gender, education of parents, experience of kindergarten) and exposure to 
Early Literacy Activity (ELA-Scale). 

Findings about gender and parental education were entirely in line with 
the literature that has consistently documented lower performance in reading 
and literacy for male students and for those coming from less educated fam- 
ilies. We reported a disadvantage of -1.70 points for boys and ad advantage 
of 2.13 points for students with a least one parent with university degree. No 
effect was found associated with kindergarten attendance net of other varia- 
bles. Controlled for the variables included in the model, the Early Literacy 
activity scale shows a linear significant effect on reading scores, with an 
increase of 0.5 points in percentage score at INVALSI reading test for each 
point increase of Literacy Activity. 
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Tab. 7 — Regression model estimating the impact of parental activities during pre- 
school on reading ability of students in grade 2 (INVALSI test) 


Reading G2 
Variables Coeff. Sign. 
Early Lit Activities 
Quantitative 0.47 0.02 
Gender 
Girl Ref Ref 
Boy -1.70 0.02 
PARENT Univeristy degree 
No Ref Ref 
Yes 2.13 0.02 
Preschool 2+years 
No Ref Ref 
Yes -1.27 0.47 
Intercept 85.73 0.00 


Tab. 8 — Regression model estimating the impact of parental activities during pre- 
school on Italian skills in grade 5 (INVALSI test) 


Literacy G5 
Variables Coeff. Sign. 
Early Lit Activities 
Quantitative 0.94 0.03 
Gender 
Girl Ref Ref 
Boy -1.27 N.S. 
PARENT Univeristy degree 
No Ref Ref 
Yes 12.83 0.00 
Preschool 2+years 
No Ref Ref 
Yes 13.53 0.00 
Score in Reading G2 (1314) 
Quantitative 0.35 0.00 


Note: The higher magnitude of coefficients with respect to model for Reading is due to the 
different scale of the score variable for Italian, WLE score, which has mean 200 and standard 
deviation 40. 
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In Table 8 results of model for predicting Italian scores at INVALSI 
test at grade 5 are summarized. This model includes all control variables 
already described for the previous regression, but also the score obtained 
by each student at reading test three years before. This variable allows us 
to take into account the “baseline” skills at grade 2 and to model the ability 
in Italian at grade 5 being equal the starting point at grade 2. This is in- 
teresting because we can detect whether the pre-school ability influences 
Italian performances, independently form the effect already had on reading 
at grade 2. From what we can see in Table 8, Reading abilities predict a 
higher score three years later, but ELA before school add another piece of 
advantage, net from that already registered in reading scores, with a cumu- 
lative effect. 

In this model Parent university degree and attending pre-school for at 
least 2 years show the highest effects, whereas gender is no longer signif- 
icant. 


6. Discussion 


The findings of this paper are consistent with previous studies demon- 
strating that the parental involvement is positively related to children’s ed- 
ucational performance: parents’ engagement with their children is strongly 
associated with better performance. 

All parents can help their children achieve their full potential by spending 
some time talking and reading with their children — even, perhaps especially, 
when their children are very young. 

This study shows that the earlier parents become involved in their chil- 
dren’s literacy practices, the more profound the results and the long-lasting 
the effects (Mullis et al., 2004). 

Furthermore, at this age, reading is among the most important subjects 
as it is transversal to acquire other skills, for this reason it is important to 
involve parents in the development of their children’s literacy (Senechal and 
LeFevre, 2002). 

Parental involvement in education should be a key focus of current poli- 
cies and programs aimed at improving the academic outcomes of students at 
risk for academic underachievement. Teachers, schools and education sys- 
tems should explore how they can help less supportive parents to play a more 
active role in their children’s education, both in and out of school. 

Policy implications should report the possibility that promoting higher 
levels of parental involvement may increase students’ cognitive outcomes, 
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and that high-quality parental involvement may help reduce performance 
differences across socio-economic groups. The development of reading pro- 
ficiency in children is perhaps the highest-ranking educational objective of 
legislators, administrators, teachers, parents, and the community. Research- 
ers who plan to examine the relationship between parental involvement and 
students’ academic achievement should pay special attention to the opera- 
tional definition and measurement of parental involvement, and should care- 
fully document such definition and measurement. 

Future perspectives of this study will include analyzing the amplifying ef- 
fect of other forms of parental involvement later on in the student’s academic 
career and a better understanding of parental characteristics that favor early 
literacy parent-child activities (e.g. work conditions). 
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3. Vocational Education and Training Courses 
in Trentino Using INVALSI Data 


by Chiara Tamanini, Mattia Oliviero, Luciano Covi 


This chapter aims to provide an in-depth analysis of the performance of 
10th grade students of vocational education and training institutes (IeFP) in 
Italian and Mathematics INVALSI tests in Trentino. It aims to: outline their 
trends compared with the other provincial school types, examine how IeFP 
students perform compared with the other upper secondary school types at 
national level (technical institutes and licei), and compare the results with 
other vocational schools in Italy, with a focus on the North-East area to iden- 
tify points of attention and foster self-improvement processes. The relevance 
of these research questions lies in the peculiar organization of the Trentino 
school system. Starting in 2011, Trentino has simplified upper secondary 
education into three main school types: “Licei” (High Schools), “Istruzione 
Tecnica” (Technical Schools), and “Istruzione e Formazione Professionale" 
(Vocational Education and Training) by gradually suppressing the "Istituti 
Professionali". As a result, IeFP witnesses an increase in the share of stu- 
dents enrolled in vocational schools (from 21% in 2005/2006 to 27% in 
2017/2018), while in the last school year the share of students in the “Licei” 
and “Tecnici” was 42% and 31% respectively. Within the Trentino school 
system, IeFp is part of a vertical curriculum which involves students aged 6 
to 16 in a continuum that, besides professional development, aims to provide 
essential performance levels as well as fundamental skills for developing ac- 
tive citizenship and civic competences. Consequently, starting in the school 
year 2012/2013 all the IeFP schools have participated in the Italian language 
and Mathematics IN VALSI tests thanks to an agreement between the Auton- 
omous Province of Trento, IPRASE and INVALSI. Therefore, it is particu- 
larly relevant to evaluate the performance of this important educational sec- 
tor, especially after the introduction of computer-based testing and INVALSI 
tests at grade 13. To investigate the research questions mentioned above, it 
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is essential to have information about the performance of students at several 
time points (longitudinal perspective). Using multilevel regression models 
(two levels: schools and students), this paper aims to examine to what extent 
the differences observed between different school types are due to students’ 
(self-)selection into upper secondary schools. This line of research is feasi- 
ble only if we have access to the students’ performance at grade 8 and 10 to 
investigate the level of association, controlling for other individual charac- 
teristics, between the two points in time for the last three school years. 


Il capitolo propone un approfondimento sugli esiti degli apprendimenti 
rilevati attraverso i test INVALSI di grado 10 della IeFP in Trentino allo 
scopo di: delinearne l'andamento rispetto agli altri indirizzi di studio pro- 
vinciali, cercare di capire come si collochino rispetto al quadro complessi- 
vo degli esiti delle prove INVALSI nel secondo anno della SSSG (indirizzi 
liceali e tecnici), confrontare i loro risultati con altri indirizzi professionali 
a livello nazionale, con particolare rifermento al Nord-Est, con l'intento di 
individuare punti di attenzione per attivare processi di auto miglioramen- 
to. L'interesse delle domande di ricerca fa leva sulla particolare organiz- 
zazione del sistema scolastico in Trentino, unico in Italia, che dal 2011 ha 
scelto di semplificare i percorsi del secondo ciclo scolastico con riferimento 
a "tre gambe": i licei, l'istruzione tecnica la IeFP, sopprimendo gradual- 
mente l'istruzione professionale. La conseguenza é stata la costante crescita 
della percentuale di studenti e studentesse che si sono iscritti ad un corso 
professionalizzante, passando dal 21% dell'a.s. 2005/2006 al 27% dell’a.s. 
2017/2018. In quest'ultimo anno scolastico la percentuale di iscritti agli in- 
dirizzi liceali é stata del 42% e quella degli istituti tecnici del 31%. Nel 
quadro scolastico del Trentino gli istituti di IeFP rientrano nell'impianto 
complessivo dei Piani di Studio Provinciali e realizzano il curricolo vertica- 
le dai 6 ai 16 anni, unitario e integrato garantendo, accanto allo sviluppo 
di una dimensione professionale, lo sviluppo dei livelli essenziali di presta- 
zioni nelle skills fondamentali per la formazione di cittadini consapevoli e 
attivi. Di conseguenza, attraverso un accordo tra Provincia Autonoma di 
Trento, IPRASE e INVALSI, dall'a.s. 2012/2013 tutti gli IeFP partecipano 
alla somministrazione delle prove INVALSI in Italiano e Matematica. E per- 
tanto rilevante per la Provincia di Trento, ma anche per il sistema scolastico 
nazionale, cercare di fare il punto sugli esiti di questa sezione scolastica, an- 
che in relazione all'innovazione della somministrazione delle prove online e 
all'introduzione delle prove INVALSI nell'ultimo anno del ciclo secondario. 
Per poter sviluppare le domande di ricerca sopra indicate é fondamentale 
disporre delle performance scolastiche degli studenti in piu punti nel tempo 
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(prospettiva longitudinale) integrandole con le caratteristiche individuali. 
Il contributo intende infatti indagare in quale misura le differenze osserva- 
te tra gli indirizzi siano frutto di un effetto di (auto)selezione avvenuto al 
momento della scelta della scuola secondaria di II grado tramite modelli di 
regressione multilivello (due livelli: studente e scuola). Questi approfondi- 
menti possono concretizzarsi solo avendo a disposizione gli esiti degli stu- 
denti al grado 8 in modo da indagare il grado di associazione tra gli esiti al 
grado 6 e al grado 10 e la serie storica di esiti degli ultimi tre anni scolastici. 


1. Introduction 


This paper proposes an in-depth analysis of the results of the INVALSI 
(i.e. National School Assessment Programme) tests on learning in the 10th 
grade of vocational education and training institutes (Italian acronym: IeFP) 
in Trentino, in order to: 

— present their trends compared with other school types in the Province 
(licei, 1.e. grammar or general schools, and technical institutes); 

— compare their performance with the learning outcomes of vocational 
schools at national level and specifically in North-East Italy, which serves 
as the actual benchmark, in order to identify elements that might deserve 
more attention and take measures to improve, if needed. 

The research question is of interest due to the special organization of the 
Trentino school system, which is unique in Italy and which decided in 2011 
to simplify the upper secondary school cycle and keep “three legs", i.e. the 
licei, the technical institutes and the IeFP, while gradually eliminating voca- 
tional training (only) courses. As a consequence, the percentage of students 
enrolling in vocational education continued to increase and went from 2196 
in the 2005/2006 school year to 27% in 2017/2018 (this is the highest per- 
centage in Italy). In the same school year, the percentage of students in the 
licei was 4296 and in technical institutes 3196. Parallel to this, attendance of 
the former vocational training schools went down to approximately 2% (Fig. 
1). In fact, the Autonomous Province of Trento has only kept those vocation- 
al training courses that could not be included in technical education courses, 
i.e. those in the social care and health sectors. 


! This concerns the *Don L. Milani" upper secondary education provincial institute in 
Rovereto, which has vocational courses in the field of social care and health services, and the 
“I. De Carneri” private, state-recognised institute, which offers vocational courses for dental 
technicians. 
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=--Licei = «Istituti tecnici FP — ——Istituti professionali 


Fig. 1 — Percentage of students enrolled in upper secondary education, by school 
type, in Trentino, over the course of time 


Source: Our own analysis of data supplied by the Education Department of the Autonomous 
Province of Trento 


The choice made by the Autonomous Province of Trento to create a 
“three legged” system and to eliminate vocational training (only) schools 
followed a national debate on the reform of upper secondary education in 
the 2005-2009 period. This ended with the decision to keep vocational train- 
ing schools as part of the overall upper secondary education system and 
transform them into five-year courses, in line with the duration of the tech- 
nical institutes and of the licei. This entailed that three-year qualification 
courses would no longer exist as such and that they would remain only as 
“subsidiary courses", next to the educational paths offered by institutes ac- 
credited at regional level. 

In this transformation, the pre-existing tradition of and focus on a prac- 
tical and workshop-oriented type of vocational training, that is also the re- 
sult of the Austrian-Hungarian cultural legacy (Antonelli, 2013), enabled 
the Trentino education system to buck the national trend and basically be 
the only place in Italy where Vocational education and training courses 
have been strengthened, in terms of their quality, their organization and 
the number of teaching hours. A 2009 provision established that the cours- 
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es of state vocational training schools would be integrated into the new 
technical or vocational education and training courses as of school year 
2010/20117. 

As aresult, in Trentino the vocational education and training sector (IeFP) 
has become an integral part of the overall school system, is part of the second 
cycle and its educational offer is on the same level as the education provided 
in the licei and in the technical institutes’. Vocational education and training 
paths are included in the overall structure of the provincial curriculums and 
in the vertical, unified and integrated curriculum for pupils aged 6 to 16 years 
old. The development of vocational skills and the attainment of the essential 
performance levels in the fundamental skills are guaranteed, in order to edu- 
cate mindful and active citizens. 

Therefore, also in the light ofthe special attention devoted by the provincial 
school system — also following the activity of IPRASE^ — to the assessment 
of pupil learning and the monitoring of the consequences of its decisions, 
the Autonomous Province of Trento, IPRASE and INVALSI entered into an 
agreement so as to have all IeFP students participate in national tests starting 
in the 2012/2013 school year. The results ofthe INVALSI tests are indeed one 
of the instruments that can help us understand what level of proficiency has 
been reached by students in specific sectors and processes related to skills in 
the Italian language and in Mathematics, which are over-arching skills. They 
are essential both for students who start working after obtaining their second- 
ary school degree and for those who decide to continue their training/speciali- 
sation in the same segment (within the framework of four-year courses and/or 
higher education courses) or to move on to a new school path. 

Also in this case, the situation of Trentino is unique within the Italian 
scenario: no other Italian region or province performs the INVALSI tests in 
the IeFP sector in such a systematic and extended way. On the contrary, at 
national level, the IeFP system struggles to advance in terms of the amount 


? Decision No. 2220, 11 September 2009. 

? Cf. Provincial Law No. 5 of 2006 on the School System, art. 54, which defines the ed- 
ucation and training cycles in the Provincial education system and that, with reference to the 
second cycle, declares that it «includes five year educational paths as well as vocational edu- 
cation and training courses lasting three, four and five years, in line with applicable national 
laws; these courses are divided into two two-year periods plus an additional one-year course 
if the entire duration is five years». Art 36 of the same Provincial Law establishes the way by 
which vocational training paths are assigned. 

^ Since the very beginning of its activity, IPRASE has promoted the culture of assessment 
and has seen to the testing of learning, and especially INVALSI tests, for the Autonomous 
Province of Trento. 
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of standardized testing of learning outcomes (Zagardo, 2019)°. This made 
it possible in Trentino to report on the overall outcomes of the vocational 
education and training system, as was the case in 2014, when 1,335 IeFP stu- 
dents took the INVALSI tests, and in 2016, with 1,388 students (INVALSI, 
2014, 2016; Falzetti, 2015)5. Furthermore, the annual feedback on the results 
to all the provincial IeFP institutes enables each institute to constantly eval- 
uate and monitor its own outcomes. 

In the end, the special structure of the Trentino education system makes 
its secondary cycle schools special vis-à-vis the rest of the country (Arici, 
Covi and Oliviero, 2019) and this explains why we focus so much on the 
need for a thorough and rigorous analysis of the results of the INVALSI tests 
in relation to the IeFP segment. It is especially important to answer a number 
of questions: how do the outcomes of IeFP students compare to those of stu- 
dents in the other types of schools? And: what level do they achieve in terms 
of skills that are fundamental in order to be active citizens and in particular in 
skills in the native language and in Mathematics (Council recommendation 
on key competences for lifelong learning, 2018)? 

How does the choice ofthe different schools relate to one's previous skills 
and socio-economic background? What national or regional school type can 
we compare the Trentino IeFP outcomes with, since no other Italian region 
or province boasts such a high percentage of students in vocational courses? 
Do Trentino students score so well in the INVALSI tests for upper second- 
ary schools (INVALSI, 2018, 2019) due to the special organization of their 
education system or can we generalize this evidence and include IeFP as 
well? Can these outcomes improve? In short: What advantages derive from 
the decision made in Trentino to attach such an important role to vocational 
education and training within the Trentino school system"? 


* Nevertheless, it is important to underline that INVALSI has increased its focus on voca- 
tional education, so much so that in recent years specific tests have been devised for the 10th 
grade of Vocational education and training courses (IeFP). These tests consist of tasks that 
are relevant for the vocational environment while their level of difficulty is the same of the 
tests administered to pupils attending the second year of the other upper secondary schools. 
Their intent is to assess the attainment of those fundamental skills in the Italian language and 
in Mathematics that every young person needs to have at the end of compulsory schooling. 

* Since the Trentino school situation is special and unique, obtaining the results for IeFP 
courses entails that a specific agreement be made each time with INVALSI, while results are 
directly reported to each school by INVALSI. 

7 This is also true in the light of the continuous decrease in the percentage of students 
enrolling in the first year of vocational training (only) schools, that went from almost 19% of 
the total to just over 13%. 
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These are the important questions this paper strives to provide an answer 
to (albeit not a final and exhaustive one), taking also into consideration the 
fact that the special autonomy enjoyed by the Province of Trento permits 
“to test” solutions that can turn out to be of interest also for other areas in 
the long run*. Therefore, analysing the learning outcomes in the IeFP sector 
based on the INVALSI test results can prove relevant not only for the Prov- 
ince of Trento but also in view of what can be adopted more in general and 
at national level, especially in relation to the innovation of administering 
tests online and to the introduction of national tests in the last year of upper 
secondary school. 


2. The effects of the interaction of one's social background and 
prior performance 


Given this scenario, the transition from lower to upper secondary educa- 
tion represents a key moment in one's educational career in Italy since one's 
future occupational outcomes and life chances are significantly influenced by 
the type of education attained. 

Previous research on the topic shows that in Italy students’ educational 
choices are influenced by the educational background of their parents. Chil- 
dren of highly educated parents have higher chances to enrol in academic 
tracks, while those with low parental education show higher transition rates 
to technical and vocational schools (Schizzerotto and Barone, 2006; Triventi 
and Trivellato, 2009; Contini and Scagni, 2013; Ress and Azzolini, 2014; 
Contini and Triventi, 2016). 

In the light of the differences in performance we observed in students, 
based on their upper secondary education, it seems relevant to investigate 
whether social-background differentials in transition rates to both general 
schools and vocational training courses change at different levels of com- 
petence. Specifically, the extension of the analysis to vocational training 
courses is an important contribution, because social-background differenc- 
es are particularly pronounced for the enrolment in vocational programmes 
(Barone, 2012; Ress and Azzolini, 2014). Moreover, our study contributes 
to a better understanding of this topic by analysing two different types of 


* C. Tamanini highlighted these aspects in “Le prove INVASI nell'a.s. 2016/17 nella 
Provincia di Trento: i risultati di un anno che conclude un ciclo" (The INVALSI tests in the 
2016/17 school year in the Province of Trento: the results of a year that completes a cycle), pp. 
6-9, retrieved on March, 11, 2021, from www.iprase.tn.it (https://bit.ly/39IaexR). 
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licei separately: those that specialise in scientific subjects and in classical 
studies on the one hand and those that focus on foreign languages, human 
sciences and the arts on the other. The analysis the Province of Trento where 
vocational education and training courses are a significant component of the 
education system could also provide important information for the entire 
national education system (Ress and Azzolini, 2014). 


3. Data and methods 


To this purpose, we used data from INVALSI about the whole student 
population enrolled in the 10th grade to shed light on such an important 
branch of the Trentino education system in two distinct ways. First, we 
looked at the differences in terms of test scores in Trentino in relation to the 
different school types and compared Trentino vocational education and train- 
ing institutes (IeFP) with the vocational schools (Istituti professionali) in Ita- 
ly at large and in North East Italy. Second, our work examined the impact of 
the previous performance of students (in the 8th grade) on the transition from 
lower to upper secondary education and considered their social background. 

Data included standardized test scores in Italian and Mathematics and infor- 
mation on the students’ family background. We used the SIDI (i.e. Education 
Information System) code to link the students’ scores in the 8th grade, in school 
year 2016/2017, with their scores in the 10th grade, in school year 2018/2019. 
By doing so we included in our analysis only those students who attended 
school regularly from grade 8 to 10, i.e. those who did not fail a year. Table 1 
shows the percentage of students included in the analysis by type of school. 


Tab. 1 — Number of students included in the analysis 


Type of school Students for whom information about their school % of students 
choice was available, included in the analysis 
Scientific/classical licei 1,099 92.0 
Other types of licei 927 81.1 
Technical institutes 1,228 80.3 
IeFP 800 57.5 


We analysed data using multinomial logistic regressions and the results 
are presented in terms of predicted probabilities separately by gender’. The 
dependent variable for the transition from lower to upper secondary educa- 


? See Tab. A1 and Tab. A2 in the appendix for the descriptive statistics. 
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tion is measured by the type of school each student is enrolled in. Specifi- 
cally, four types of schools are included: Scientific licei — and classical licei 
only for the Italian language test — other types of licei, technical institutes 
and IeFP (vocational education and training courses). The crucial independ- 
ent variable we included in the analysis is the students’ INVALSI test scores 
in the 8* grade, which act as a proxy for prior performance. Given our in- 
terest in investigating the possible effects of one's social background on the 
type of school chosen, we used the information on parental education taking 
into consideration the highest level of education reached by any of the two 
parents. We coded this variable into three categories: lower secondary edu- 
cation, upper secondary education, and tertiary degree. 


4. Descriptive results 


We first examined differences in the various types of schools in the Prov- 
ince of Trento. Fig. 2 and Fig. 3 show the performance in Italian and Mathe- 
matics of the same students in the 8^ and 10^ grade in relation to the school 
type they chose. As we might expect, students show consistent differences 
based on the types of upper secondary schools they chose. In particular, re- 
sults show that students with the highest performance enrol in scientific licei. 
However, what is more interesting here is that we can observe a considera- 
ble reduction in the differences in the test scores in Italian and particularly 
among students enrolled in vocational education and training courses in the 
10^ grade; while in Mathematics the gap between the licei and the technical 
institutes and IeFP schools remains quite large. 

The second step entailed comparing the performance of students in voca- 
tional education and training courses in Trentino with the national context. 
Given the difference in their organizational structure, we compared the re- 
sults of IeFP with those of vocational schools in Italy and in the North-East 
of Italy in 2018 and 2019"°. The results (Fig. 4) show that vocational educa- 
tion and training courses in Trentino are comparable with vocational schools 
in terms of performance and, in particular, in 2019 they showed higher test 
scores both in Italian and Mathematics. 


10 For this comparison the results include all the students in the 10th grade and not only 
students for whom we have information on their type of school as well as their test scores in 
the 8th grade. 
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Fig. 2 — INVALSI Italian test scores in the Province of Trento (95% confidence inter- 
val) in the 8" and 10" erade by type of school 


Source: Our own analysis of data from INVALSI, 2017 and 2019 
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Fig. 3 - INVALSI Italian test scores in the Province of Trento (95% confidence inter- 
val) in the 8th and 10th grade by type of school 


Source: our own analysis of data from INVALSI, 2017 and 2019 
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Fig. 4 — Average test scores in Italian and Mathematics by geographic area 


Source: our own analysis of data from INVALSI, 2018/2019 


5. Results 


In this section, we examine the role of the students’ social background and 
level of competence in the transition to different programmes of upper second- 
ary education. The results are presented in the form of predicted probabilities 
from multinomial logistic regressions!! for upper secondary school transition. 
The analyses include the interaction between parental education and INVALSI 
test scores in Italian in the 8th grade, separately for males and females. 

The results for Italian (Fig. 5) clearly show that for all social groups, and 
for both girls and boys, the higher the level of competence, the higher the 
likelihood that they enrol in the licei and the lower the likelihood that they 
enrol in Vocational education and training courses. However, the graph also 
shows that social disparities in relation to the enrolment in the licei do not 
disappear among top-performing students. 

For instance, among low performing students, the sons of tertiary-edu- 
cated parents show a 20 percent probability to enrol in licei compared with 


1 See Tab. A3 and Tab. A4 in the appendix for details of multinomial logistic regressions. 
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less than 10 percent among sons of parents who completed lower secondary 
education. This difference becomes even larger among better-performing 
students. Sons of highly educated parents, who achieve high scores, have 
almost 80 percent chances to enrol in licei, whereas equally high-performing 
students whose parents have attained lower secondary education reach 20 
percent. The trend is similar for females. 

However, what considerably differentiates females and males is the enrol- 
ment in the other types of licei — which include courses focussing on foreign 
languages, human sciences, and the arts — and, to a lesser extent, in the tech- 
nical institutes. Specifically, the category of the other types of licei appears to 
be a highly gender-segregated programme, with a percentage of female stu- 
dents enrolled in these programmes totalling 82%. The probability to enrol in 
one of these programmes is particularly high among low-performing females 
with highly educated parents. The probability for this group is approximately 
50 percent while low-performing females whose parents are less educated 
show a probability of approximately 20%-30%. Among males, the risk of 
transition is negligible. 


Liceo scientifico/classico, MaschioLiceo scientifico/classico, Femmina Altri licei, Maschio Altri licei, Femmina 


leFP, Femmina 


Probability 


Fig. 5 — Multinomial logistic regression of probability of choice of the type of upper 
secondary school in the 10" erade: predictive margins from a model with interaction 
between parental education and INVALSI test scores in the 8" erade (Italian) 


Source: our own analysis of data from INVALSI, 2017 and 2019 


The results also show that high scores in the INVALSI test almost com- 
pletely eliminate the likelihood of vocational training course enrolment for 
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all groups. For example, the probability of choosing vocational training 
courses for boys and girls who got an average score (200) on the Italian test 
is approximately 10%, if their parents completed a tertiary education while 
it is approximately 40% if their parents attained lower secondary education. 

The probability of attending technical institutes differs little among 
low-performing students with different social backgrounds, whereas among 
high-performing students it is larger for those with low educated parents but 
in general lower among females who are more likely to enrol in the other licei. 

The results for Mathematics (Fig. 6) are very similar to the results for 
Italian, the differences can be summarized in three major points. 


Liceo scientifico, Maschio Liceo scientifico, Femmina Altri licei, Maschio Altri licei, Femmina 
_ SSS MÓ—Á—Ó I| MM [| raÀ——ÀÀ—| 


leFP, Femmina 
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Fig. 6 — Multinomial logistic regression of probability of choice of the type of upper 
secondary school in the 10" erade: predictive margins from a model with interaction 
between parental education and INVALSI test scores in the 8" erade (Mathematics) 


Source: Our own analysis of data from INVALSI, 2017 and 2019 


First, the differences in the enrolment in the licei are smaller among stu- 
dents with different social backgrounds when compared with the Italian test 
scores. Specifically, the differences are smaller among top-performing stu- 
dents: the chances to enrol in the licei are higher for students with less-ed- 
ucated parents — approximately 50% — if we compare them with the Italian 
score — approximately 30%. Second, the probability to enrol in other types of 
licei in the case of low test scores 1n Mathematics is higher for females with 
highly educated parents — 70% vs. 50% for the Italian score. This is counter 
balanced by a low probability to enrol in technical institutes and vocational 
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education and training courses. Third, top-performing students in Mathemat- 
ics are more likely to enrol in the licei and to a much lesser extent in techni- 
cal institutes, with small differences between different social backgrounds. 


6. Conclusions 
Fig. 7 below provides a graphical representation of the peculiarity of the 


Trentino school system in comparison with the geographical benchmark of 
North-East Italy and Italy as a whole. 
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Fig. 7 — Percentage of students enrolled in upper secondary education by school 
type (s.y. 2017/2018) 


Source: Our own analysis of data supplied by the Education Department of the Autonomous 
Province of Trento 


After completing lower secondary school, students in the Province of 
Trento can choose between three different types of schools: licei, technical 
institutes and vocational education and training courses (IeFP). Our analy- 
sis confirms previous findings by showing that the transition from lower to 
upper secondary education is still largely influenced by the students’ family 
background. Students with lower INVALSI scores and parents who did not 
attend tertiary education have a higher probability to enrol in IeFP schools, 
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while students with higher test scores and with parents with a tertiary de- 
gree are more likely to enrol in classical or scientific licei". As evidenced in 
several research works", it is especially the family’s socio-economic back- 
ground and gender that influence the choice of schools. 

The Trentino school system, therefore, enables students who already en- 
counter problems in lower secondary education to continue their general ed- 
ucation while attaining a vocational qualification. What about the INVALSI 
test results of IeFP female and male students in the 10th grade compared with 
the outcomes of their counterparts in the licei and technical institutes? Fig. 8 
and 9 below show more clearly what already emerged from Fig. 2 and 3, with 
reference to the test scores in the 2017/2019 period. 
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Fig. 8 — Average test scores in Italian 


Source: Our own analysis of data from INVALSI, 2017 and 2019 


? Other investigations of the learning outcomes at the end of primary school, including 
the results of the school-leaving state exam, showed the same trend (Tamanini, 2007) and, once 
again, the relation between school evaluations and the scores in national tests is evidenced. 

B See, among others, the Alma Diploma analyses of the profile of school graduates. 
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Fig. 9 — Average test scores in Mathematics 


Source: our own analysis of data from INVALSI, 2017 and 2019 


The test results of IeFP students are lower than their counterparts attend- 
ing the licei or technical institutes, but we notice a smaller disadvantage in 
the Italian language score and a marked improvement of this score compared 
with the 8th grade. The trend should be monitored over several years; nev- 
ertheless, there is clearly room for improvement in the learning outcomes 
for Mathematics and a slightly positive trend for the Italian language, also 
in light of the fact that the percentage of students of foreign origin attending 
IeFP is higher, especially among first generation migrant children (14.8% vs. 
6% in the licei and technical institutes). 

With regard to the school effect of the IeFP, it will be important to deepen 
the issue with further study to better understand the quality of this upper 
secondary school path (Ricci, 2020). 

Differences between IeFP results and other school orders in terms of le- 
vers are also confirmed by the trends outlined in the PISA international sur- 
veys (cf. Fig. 10), which indicate a progressive improvement in the IeFP 
sector in the past ten years, despite an evident differential compared with the 
licei and the technical institutes. 
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Results also highlight that Trentino IeFP schools show higher score in 
Italian and Mathematics compared with the national average of vocational 
schools in the 2017/2018 and 2018/2019 school years (cf. Fig. 4) and that 
the same applies when the Trentino system is compared with the area of 
North-East Italy, even though in this case the difference 1s smaller. Vis-à-vis 
North-East Italy, in 2018 only results in Mathematics were identical and in 
2019 they were back to being better. Trends will need to be monitored over 
the years; nevertheless, this result too might indicate that there is room for 
improvement in the teaching and learning of Mathematics in the IeFP sector. 

Moreover, a positive aspect of the “three legged” organization of the 
Trentino school system is the question of the drop-out rates, that clearly de- 
creased after introducing the new model (from 11.7% in 2012 to 6.7% in 
2018), thereby making Trentino one of the best areas in Italy in terms of con- 
trolling explicit and implicit early school-leaving (cf. Figure 11 and Ricci, 
2019). 
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Fig. 11 — Explicit and implicit early school-leaving in Italian regions 


Source: Ricci (2019) 
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Indeed, enhancing the IeFP system on a socio-cultural level and making 
it an integral part of the upper secondary education and training cycle of the 
Province of Trento has allowed to prevent and limit drop-out phenomena, 
especially among low performing students at the end of the first cycle, while 
increasing their level in the fundamental skills. This increase is guaranteed 
over the course of time, with no “ceiling effect" thanks to: 

— the possibility to change type of school during or after the first two years 
of upper secondary school, which coincide with the end of compulsory 
schooling; 

— the progressive construction over the years of a system which has further 
developed vocational qualification three-year programmes and has added 
a fourth year, at the end of which a vocational degree is obtained, an 
annual course for the vocational education State exam (called CAPES in 
Italian), which makes it possible for students completing the fourth year 
to take a special State exam based on a protocol signed by the Italian 
Ministry for Education, University and Research and the Autonomous 
Provinces of Trento and Bolzano and, more recently, the activation of 
higher vocational education courses, i.e. non-academic tertiary education 
programmes, formally equal to the Higher technical institutes pursuant to 
article 8 of Ministerial decree No. 8327 of 7 September 2011, for students 
who completed the fourth year in an IeFP school and who obtained a 
vocational degree. 
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Appendix 


Tab. Al — Descriptive statistics for the Italian test 


Scientific Other Technical VTC Total 
and Classic Lyceums Schools 

Lyceums 
Variables Mean Sd Mean Sd Mean Sd Mean Sd Mean Sd 
Italian Score 10" grade 234.1 32.0 215.5 33.0 203.8 32.8 188.5 36.1 211.8 36.7 
Italian Score 8" grade 229.4 32.0 210.2 31.6 199.9 31.0 170.4 31.6 203.5 37.7 
Female 0.50 0.50 0.83 0.38 0.36 0.48 0.39 0.49 0.51 0.50 
Lower sec. 0.10 0.30 0.20 0.40 0.26 0.44 0.45 0.50 0.24 0.43 
Upper sec. 0.37 0.48 0.44 0.50 0.46 0.50 0.34 0.48 0.41 0.49 
Tertiary degree 0.40 0.49 0.23 0.42 0.13 0.33 0.05 021 021 0.40 
N 1,099 927 1,228 800 4,054 
Tab. A2 — Descriptive statistics for the Mathematics test 

Scientific Other Technical VTC Total 

Lyceums Lyceums Schools 
Variables Mean Sd Mean Sd Mean Sd Mean Sd Mean Sd 
Math Score 10" grade 249.3 32.8 206.2 31.2 218.5 31.7 193.3 294 2189 37.1 
Math Score 8"grade 246.1 36.3 207.0 33.4 215.6 34.3 185.5 34.6 215.2 40.8 
Female 0.50 0.50 0.83 0.38 0.36 0.48 0.39 0.49 0.51 0.50 
Lower sec. 0.10 0.30 0.20 0.40 0.26 0.44 0.45 0.50 0.24 0.43 
Upper sec. 0.37 048 0.44 0.50 046 0.50 0.34 048 0.41 0.49 
Tertiary degree 0.40 0.49 0.23 0.42 0.13 0.33 0.05 021 0.21 0.40 
N 950 1,058 1,223 800 4,031 
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Tab. A3 — Multinomial logistic regression on the choice of upper secondary school 
in the 10" grade versus Technical track with interaction between parental education 
and INVALSI test scores in the 8" grade (Italian) 


Variables Scientific and Other Lyceums VTC 
Classic 
Lyceums 
. 0.02*** 0.00 -0.03*** 
Italian Score 10" grade 
(0.00) (0.00) (0.00) 
Highest Parental Education (ref. Lower sec.) 
o S d 0.06 -0.58 -0.85 
er Secondar 
PR d (0.99) (0.79) (0.72) 
dedu n 1.12 1.05 -0.50 
ia n 
Hee eer (1.13) (1.03) (1.34) 
Upper secondary * Italian Score 10^ grade boh en ed 
EP T 5 (0.00) (0.00) (0.00) 
: . 0.00 -0.00 -0.00 
Tertiary Degree * Italian Score 10" grade 
(0.01) (0.00) (0.01) 
Female 
(0.10) (0.11) (0.10) 
ta -0.29 -0.09 -0.25 
1* generation immigrant 
(0.37) (0.28) (0.26) 
TP -0.30 -0.03 -0.22 
2"! generation immigrant 
(0.21) (0.19) (0.18) 
Grade repetition (ref. never) 
-].89*** 0,52% IPS 
At least one year 
(0.57) (0.27) (0.21) 
-6.08*** -2.46*** 5.39%** 
Intercept 
(0.84) (0.63) (0.52) 
Observations 4,054 4,054 4,054 


Standard errors in parentheses; *** p < 0.01, ** p < 0.05, * p < 0.1. 


Source: Our own analysis of data from INVALSI, 2017 and 2019 


68 


ISBN 9788835131076 


Tab. A4 — Multinomial logistic regression on the choice of upper secondary school in 
the 10th grade versus Technical track with interaction between parental education 
and INVALSI test scores in the 8th grade (Mathematics) 


Variables Scientific Other VTC 
and Classic Lyceums 
Lyceums 
. 0.02*** -0.01* -0.03*** 
Mathematics Score 10" grade 
(0.00) (0.00) (0.00) 
Highest Parental Education (ref. Lower sec.) 
U S d 0.45 -0.75 -1.62** 
er Secondar 
PP d (0.95) (0.72) (0.70) 
Tertiary D 1.95* 0.47 -1.70 
ertiary Degree 
ric (1.08) (0.92) (1.25) 
U dar * Italian $ 10* grad 0.00 0.01* 0.00 
er seconda alian Score rade 
dd si 5 (0.00) (0.00) (0.00) 
: . 0.00 0.00 0.00 
Tertiary Degree * Italian Score 10" grade 
(0.00) (0.00) (0.01) 
0.84*** ZIE -0.13 
Female 
(0.10) (0.10) (0.10) 
ta -0.29 -0.24 -0.08 
1* generation immigrant 
(0.38) (0.28) (0.25) 
ree -0.25 -0.17 -0.20 
2"! generation immigrant 
(0.21) (0.18) (0.18) 
Grade repetition (ref. never) 
eas -1.86*** 0.41 1,16*** 
ast one year 
ic (0.62) (0.27) (0.22) 
-6.31*** -0.78 2.99*** 
Intercept 
(0.83) (0.60) (0.53) 
Observations 4,031 4,031 4,031 


Standard errors in parentheses; *** p < 0.01, ** p < 0.05, * p < 0.1. 


Source: Our own analysis of data from INVALSI, 2017 and 2019 
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4. Finding Forrester. An empirical study on foreign 
students penalty in Italian high schools 


by Lorenzo Maraviglia 


Knowledge doesn t start from perceptions or observations or the collection 
of data or facts but it starts, rather, from problems. 
(K.R. Popper) 


According to updated projections (ISTAT, 2020), the number of students 
attending Italian primary and secondary schools will sharply decline in the 
next few years; in the meanwhile, the share of immigrant pupils is about to in- 
crease constantly. Successful educational integration of the latter will become 
pivotal for the prospective performance of the overall social and economic 
system. This issue is part of the wider challenge of tackling diversity and 
growing complexity within the framework of our educational system. In such 
context, great attention should be paid to divergent clues. In this perspective, 
following Kiss (2013) contribution and drawing on INVALSI data, Pini and 
Triventi (2016) have shown that, individual test scores being equal, immi- 
grant students tend on average to receive lower midterm marks than natives 
in Italian and Math. In a school where enrollment of foreign born teachers 
(other than language teachers) is still far to come, such signal must be consid- 
ered seriously and with scientific attitude. The present work contributes to the 
discussion leveraging the extraordinary extension and coverage of INVALSI 
data in order to describe geographic (at NUTS3 level) and typological (with 
reference to school features) variation in potential discriminatory evaluation 
of immigrant students. Although exploratory, such work is a necessary prem- 
ise for more structured causal analysis and identification and for the objective 
assessment of the extension of a possible emerging problem. More specifi- 
cally, we show that risk of discrimination is higher in lyceal institutions and, 
more generally, in schools where students are strongly selected according to 
social features (positive selection). 


Alla luce delle proiezioni piu recenti (ISTAT, 2018), il numero di studenti 
delle scuole primarie e secondarie del nostro Paese é destinato a ridursi 
drasticamente nei prossimi anni. Contemporaneamente aumentera la quota 
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di alunni immigrati e/o figli di immigrati e l'integrazione di tali soggetti 
assumerà un’importanza centrale per la performance del sistema sociale 
ed economico. La questione e parte della piu ampia sfida relativa a come 
affrontare condizioni di crescente diversità e complessità all'interno del si- 
stema educativo nazionale. A questo proposito, traendo ispirazione dal la- 
voro di Kiss (2013) ed attingendo ai dati INVALSI, Pini e Triventi (2016) 
hanno mostrato che, a parità di risultati nei test somministrati, gli studenti 
immigrati nel nostro Paese tendono ad avere una valutazione mediamente 
piu bassa rispetto ai nativi in Italiano e Matematica nelle pagelle del primo 
trimestre. In una scuola in cui l'inserimento di insegnanti stranieri/immigra- 
ti (fatta eccezione per i docenti di lingue) é ancora di là da venire, questo 
segnale dovrebbe essere considerato con attenzione e con attitudine scien- 
tifica. Il presente lavoro contribuisce alla discussione sfruttando la straor- 
dinaria estensione e copertura dei dati INVALSI, con l'intento di ricostruire 
la variabilità geografica (a livello di province) e tipologica (con riferimento 
alle caratteristiche delle singole scuole) di un eventuale valutazione discri- 
minatoria degli studenti stranieri. Seppur esplorativa, questa analisi é una 
premessa necessaria per ricerche piu strutturate, volte ad indagare il tema 
sotto il profilo dei processi causali. Più specificamente, mostreremo che il 
rischio di discriminazione è più elevato nei licei e, in generale, nelle scuole 
in cui gli studenti tendono ad essere fortemente selezionati in base a caratte- 
ristiche di status (selezione positiva). 


1. Introduction 


Foreign students account for a growing share of Italian school population 
(MIUR, 2019). In some of the most productive areas of the country, the in- 
cidence of this group exceeds 20% among primary school pupils and such 
figure is expected to increase in the future. Following this epochal change, 
concerns begin to emerge about the model of integration of these students. 
The issue is exacerbated by the fact that almost all teachers — at any school 
level — are native Italians, often lacking specific training on this subject. 

The risks of possible, subtle, forms of penalization of students belonging 
to ethnic or national minorities have been highlighted in the international 
literature. The problem has been thoroughly discussed in Anglo-Saxon coun- 
tries (Heckman and Cuhna, 2010) and Northern Europe (Kiss, 2013). Build- 
ing on this strand of literature, and taking advantage of the great cognitive 
potential of INVALSI data, Pini and Triventi (2016) find that «INVALSI 
score being equal, foreign students tend to receive lower marks than their 
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Italian peers, especially in primary school [...] foreign students born in Italy 
are generally less penalized than those born abroad but the differences are 
small [...] the under-assessment varies according to the subject, being more 
pronounced in Mathematics than in Italian». 

Obviously, this statement requires further investigation. Thanks to their 
standardized nature and controlled methods of administration, INVALSI 
scores provide an important test for the ability of the national school system 
to integrate foreign students, guaranteeing the same chances of success re- 
served for their Italian mates. The fact that a possible penalization tends to 
appear more pronounced in lower grades could depend on heterogeneity of 
teachers', but also on differences in qualitative composition of foreign stu- 
dents, due to the fact that national structure of migration flows has changed 
over time; or it could be a consequence of the existing tracking mechanisms, 
resulting in more homogeneous secondary classes. 

This empirical study places itself in the wake outlined by Pini and 
Triventi and tries to develop some of the ideas raised by them, in particular 
by exploring some aspects of territorial (between local systems of our coun- 
try) and school (between types of high schools) variability of gaps between 
INVALSI results and curricular marks. 


2. Data 


The data that form the subject of our empirical analysis come from the 
INVALSI questionnaire administered in the second classes of Italian high 
schools in the year 2017/2018. Specifically, we will focus on the compari- 
son between students’ results in the Math test (INVALSI score in MAT) and 
marks received at the end of the first quarter (first term mark in MAT). The 
distribution of the two variables in the sample is shown in the upper panels 
of Figure 1. 

Students distribution by school type (bottom-left panel) exhibits a certain 
balance’ whereas that by citizenship shows a clear prevalence of the Italian 


' For example, this could depend on the fact that primary school teachers receive less 
training on subjects that help in managing linguistic and cultural differences. However, this 
point is controversial and we will not address it specifically in the following. 

? Lyceums are divided into those implementing a scientific curriculum (LIC_S) and all 
the others (LIC. A). Unfortunately, the analysis suffers from the high level of aggregation with 
which information about the school attended by students is made available in the micro-data 
accessible to researchers outside INVALSI. 
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component’. Ethnic imbalances are highly variable across schools and terri- 
tories — a point that will be taken up in the discussion. 


First term grade in MAT (oral) INVALSI score in MAT 
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Fig. 1 — Descriptive statistics 


Source: elaboration on INVALSI micro-data 


3 Students for whom nationality is not available (coded *NA") were treated as a separate 
category in the regression models; the results with respect to this group are not presented in 
the following tables and graphs. The overall sample size (including Nas) is 452,792 cases. 
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3. Analysis 


Our analytical strategy is based on regression. We start by estimating a lin- 
ear model in which the mark in Math received at the end of the first quarter is 
regressed on students’ citizenship, type of school attended and INVALSI score: 


mark, = a + b * citizenship, + c * score; + d * school; + e; 


with e; being the error component assumed to be normally distributed, with 
zero mean and s standard deviation. 

The predictor we are most interested in is, obviously, students' citizen- 
ship. Since citizenship is a categorical variable, the regression coefficients 
associated with it provide an estimate of the average difference in Mathe- 
matics marks between Italian students (reference category) and, respective- 
ly, first and second generation foreign students, after controlling for type of 
school attended and INVALSI score. Territorial variability is not taken into 
consideration at this level, but will be discussed later in the analysis. The 
results (parameters) are shown in table 1. Instead of just the final parameters, 
we report the results of a series of models of increasing complexity, starting 
from the intercept-only model (Mod. 0) and ending with the one including 
all main effects plus an interaction term between citizenship and school type 
(Mod. 4). This allows to appreciate the variations in the coefficients — in 
particular in those associated with citizenship — as new stratification criteria 
are added to the analysis. 

The difference in the average oral Math mark between Italian and foreign 
students is about 1/3 of a point (Mod. 1). This result is obtained before con- 
trolling for any other variable, therefore it has a purely descriptive value. In 
the second model (Mod. 2) the INVALSI score is added to the predictors; this 
allows to adjust for heterogeneity in the actual average Mathematics skills 
of the two groups, thus revealing a possible penalization of foreign students. 

The point is delicate and potentially controversial. Assuming that the 
INVALSI score captures the true state of things, while the judgment ex- 
pressed by teachers might be affected by prejudices, is a very strong posi- 
tion which we do not intend to support. We only want to draw attention to 
an aspect which, where confirmed by the analysis, should raise doubts and 
stimulate discussion within the community of scholars and teachers. 

Therefore, we simply record that, INVALSI score being equal, the aver- 
age difference between marks of Italian and foreign students shrinks to about 
one tenth of a point (0.10 per first generation, 0.15 for second generation), 
while remaining statistically significant. 
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Tab. 1 — Regression models coefficients 


Mod_0 Goodness-of-fit statistics 
Parameters Est. S.E. N. obs AIC R-squared 
(Intercept) 6.04 0.00 452,795 1,612,962 

Mod 1 Goodness-of-fit statistics 
Parameters Est. S.E. N. obs AIC R-squared 
(Intercept) 6.09 0.00 452,795 1,609,688 0.01 
Italian (reference) 
First generation -0.38 0.01 
Second generation -0.31 0.01 

Mod 2 Goodness-of-fit statistics 
Parameters Est. S.E. N. obs AIC R-squared 
(Intercept) 6.05 0.00 452,795 1,521,648 0.17 
Italian (reference) 
First generation -0.10 0.01 
Second generation -0.15 0.01 
INVALSI mat score (centered) 0.13 0.00 

Mod 3 Goodness-of-fit statistics 
Parameters Est. S.E. N. obs AIC R-squared 
(Intercept) 5.73 0.00 452,795 1,506,594 0.20 
Italian (reference) 
First generation -0.11 0.01 
Second generation -0.14 0.01 
INVALSI mat score (centered) 0.17 0.00 
Science Liceum (reference) 
Liceum (other) 0.61 0.01 
Technical school 0.16 0.01 
Professional school 0.58 0.01 

Mod 4 Goodness-of-fit statistics 
Parameters Est. SE. N. obs AIC R-squared 
(Intercept) 6.11 0.00 452,795 1,506,415 0.20 
Italian (reference) 
First generation -0.14 0.03 
Second generation -0.22 0.02 
INVALSI mat score (centered) 0.17 0.00 
Science Liceum (reference) 
Liceum (other) 0.61 0.01 
Technical school 0.16 0.01 
Professional school 0.54 0.01 
Interaction terms 
First gen. * Liceum (0.) -0.11 0.03 
Second gen. * Liceum (o.) 0.02 0.03 
First gen. * Technical 0.04 0.03 
First gen. * Professional 0.16 0.03 
Second gen. * Technical 0.11 0.03 
Second gen. * Professional 0.15 0.03 


Source: elaboration on INVALSI micro-data 
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In the next model (Mod. 3), school type is introduced among the predic- 
tors. School is an important element because, at high school level, Italians 
and foreigners tend to make different choices — foreigners are strongly un- 
der-represented in lyceums — and a possible inclination to penalize the latter 
could vary according to the type of institution attended. To fully grasp this 
point, we do not limit ourselves to considering the main effects of the type of 
school but we also introduce the interactions between school and citizenship 
(Mod. 4). Given the difficulty of interpreting the coefficients of a model with 
interactions, we plot results (Fig. 2). 
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Fig. 2 — Regression model with interaction term between citizenship and school type 


Source: elaboration on INVALSI micro-data 
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As it can be seen, the penalty factor is marked in lyceums, both scientific and 
other type, and more limited in technical and professional schools; the latter are 
also the types of schools in which foreign students are more represented. There- 
fore, the penalty tends to be maximum where presence is minimal, and vice versa. 


4. Territorial heterogeneity 


So far, the issue of geographical heterogeneity has not been addressed in 
our models. It is however clear that, since school outcomes vary greatly from 
place to place (and the same holds for characteristics of immigrant popula- 
tion), the topic deserves attention. The following is a first attempt to organi- 
cally include the territorial dimension in the systematic study of the penali- 
zation of immigrant students. The choice of the territorial level of analysis is 
extremely important. In the majority of available studies, the students popu- 
lation is generally stratified by NUTSI or, at most, by NUTS2 region. In this 
way, however, much of the local variability that characterizes our country is 
lost. A very promising type of geographical aggregation, made possible by 
the level of detail with which the INVALSI micro-data are provided, is that 
of the Local Market Areas (LMAs) defined by ISTAT on the basis of the cen- 
sus results. Local Market Areas are groups of adjoining municipalities which 
contain within their aggregated borders a predefined share of commuting 
flows. In practice, LMAs are relatively homogeneous socio-economic areas, 
identifying sort of local labor markets; given the connection between char- 
acteristics of the labor market, composition of the immigrant population and 
training choices made by young people and their families, it can be under- 
stood how relevant this dimension is for the purposes of our study. 

To encompass the territorial element in our analysis, we adopt a multilev- 
el framework by estimating the following model in which both the intercept 
(a) and the angular coefficient of citizenship (b) are allowed to vary accord- 
ing to the Local Market Area: 


= * citi i * 
mark;= a;+ b;* citizenship;* c * score; u;+ e; 


where ; index students residence in one of over six hundred LMAs identified 
by ISTAT. 

Given the complexity of the model and the large number of parameters 
implied by it, we provide a summary representation of the results by reporting 
average differences in marks between Italian and foreign students, INVALSI 


score being equal, on a map. 
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Fig. 3 — Differences in Math marks between Italian and foreign students, INVALSI 
being equal; geographic distribution at LMAs level 


Source: elaboration on INVALSI micro-data 
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On average, the penalization of immigrants tends to be lower in the 
North-East (Veneto, Emilia Romagna) and higher in Central and Southern 
Italy. There are however exceptions which certainly deserve to be investigat- 
ed in more detail. In general, what can be drawn from the observation of the 
map is that the phenomenon has a significant level of geographic correlation; 
this points to the possible existence of contextual factors (social, economic, 
cultural) that can interact with some more general elements depending on the 
way teachers are selected and trained". 


5. Discussion 


Pini and Triventi conclude their study (2016) observing that they cannot 
rule out that «some kind of “statistical discrimination” is at work: professors, 
in case they are uncertain about how to evaluate foreign students, could be 
influenced by stereotypes ethnic, ending up penalizing them compared to 
their Italian companions». Obviously this is a strong statement; however, as 
the two authors write, this cannot be excluded in the presence of a persistent 
evaluation gap after controlling for auxiliary elements, such as INVALSI 
Scores. 

This study has tried to further explore the issue of penalization by deep- 
ening the investigation with respect to the heterogeneity of types of schools 
and territories. Especially for the latter, further development possibilities are 
foreseen, once the extraordinary opportunities deriving from the combina- 
tion of the geographical detail allowed by the data and the power of the 
multi-level instruments are adequately deployed (Gelman and Hill, 2007). 
As far as schools are concerned, a greater typological detail would certainly 
be necessary which would allow, for example, a distinction to be made be- 
tween the residual category of “other lyceums" and humanistic and artistic 
or socio-pedagogical ones. 

In the coming years, the share of young foreigners who attend high 
schools in our country is expected to grow significantly. This will take place 
in a context of generalized reduction in the number of students, following the 
demographic crisis affecting our country. In general, young foreigners will 


^ [t can be observed that North-East areas exhibit (on average) a higher incidence of for- 
eign students. However this is true even for many areas in the North-West or in the Center of 
Italy. North-East (especially Veneto and Trentino-Alto Adige) is also where average INVALSI 
scores are higher than the rest of the Country, and this could provide an additional explana- 
tion (the higher teaching quality, the lower penalization). However, we are simply suggesting 
some directions for future research, without any strong affirmative intention. 
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become an essential part of the national workforce and production potential, 
as well as an essential resource for the survival and renewal of prestigious 
high school curricula (e.g. classical lyceums and traditional scientific lyce- 
ums) who are now experiencing a marked reduction in enrollments. These 
schools are also those where, today, foreign students are less represented 
and, where present, apparently more penalized in terms of assessment. These 
aspects, perhaps, are not only empirically related but arise from a common 
substratum that deserves to be investigated. 


References 


Gelman A., Hill J. (2007), Data Analysis Using Regression and Multilevel/Hierar- 
chical Models, Cambridge University Press, Cambridge (MA). 

Heckman J., Cuhna F. (2010), Investing in our young people, IZA Discussion Paper, 
5050. 

ISTAT (2020), Rapporto annuale 2020, Edizioni ISTAT, Roma. 

Kiss D. (2013), “Are immigrants and girls graded worse? Results of a matching 
approach”, Education Economics, 21, 5, pp. 447-463. 

MIUR (2019), Gli alunni con citadinanza non italiana, Statistiche e studi del MIUR, 
retrieved on March, 11, 2021, from www.miur.gov.it. 

Pini E., Triventi, M. (2016), “Sottovalutati dagli insegnanti? L’attribuzione dei voti 
agli studenti nativi e stranieri nelle scuole italiane", Neodemos, 8 aprile. 


80 


ISBN 9788835131076 


5. Data on learning and teacher decision-making. 
An "Italian marriage”? 


by Serafina Pastore, Cataldo Scarnera, Lucia Pallucca 


Nowadays, the references to a teacher professional model that is evi- 
dence-based and data-driven have reinforced the need for teachers to avoid 
take decisions on the base of their perceptions and anecdotes. Furthermore, 
teachers are called to use data gathered from different sources in order to 
inform their teaching practice and to develop better levels of assessment and 
data literacy. Therefore, teacher data-driven decision-making rouses great 
interest. Different studies, moreover, have pointed out that 1f teachers (and 
principals) effectively use data, students’ learning achievements will im- 
prove. 

Data, especially those gathered from large-scale assessment programs, 
despite recent requests of a radical revision of their rationale and structure, 
can be used for different aims: school improvement, revision of instructional 
practice, accountability. 

If, on the one hand, it has been proved that contextual factors can influ- 
ence the use of data, on the other hand, it is true that data characteristics and 
data systems can influence how data are used. In this vein, the use of data 
from large-scale assessment programs depends on the users’ characteristics. 
Consequently, crucial elements are the teachers’ and principals’ assessment 
competences in analysing and interpreting data, in highlighting criticalities 
and problems on the school performance, and in designing what actions take 
(assessment literacy). 

Assuming that data, at macro and micro-levels, allow different uses, this 
paper reports on the first phase of a research project realized in a primary 
school in the South of Italy and aimed to implement an interim assessment 
system. Moving from an analysis of INVALSI data at micro-level (school 
year 2018/19), two different rounds of teacher professional development 
have been realized to train teachers to analyse INVALSI data, and to design 
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new assessment tests on students’ learning. Recalling the main aspects of 
this study, the paper critically reflects on how to facilitate the standardized 
assessment data use for the teaching practice. 


Irichiami a un modello professionale evidence-based e data-driven, negli 
ultimi anni, hanno costantemente ribadito la necessita di evitare che le deci- 
sioni degli insegnanti si basino su aneddoti e intuizioni. Ciò implica che gli 
insegnanti usino dati provenienti da diverse fonti per informare la propria 
pratica didattica e che, in termini piu generali, sviluppino migliori livelli di 
assessment e data literacy. // decision-making basato sui dati desta, oggi, 
particolare attenzione. Diversi studi, del resto, hanno dimostrato come l'uso 
efficace dei dati da parte degli insegnanti (e dei dirigenti) possa indurre 
un miglioramento reale della scuola in termini di apprendimento maturato 
dagli studenti. I dati, specie quelli ricavati dalle rilevazioni su vasta scala 
sull'apprendimento degli studenti, al di là delle istanze di revisione profonda 
di tali programmi di valutazione, possono essere utilizzati per finalità diver- 
se: miglioramento della scuola, azione didattica, accountability. Se, da un 
lato, é stato dimostrato come gli aspetti contestuali possano influire sull'uso 
dei dati, è anche vero che le caratteristiche dei dati e dei sistemi di dati pos- 
sono influenzare le modalità di utilizzo dei dati stessi. Pertanto l'uso dei dati 
ricavati dalle rilevazioni su vasta scala sull'apprendimento degli studenti 
dipenderebbe dalle caratteristiche degli utilizzatori e, pertanto, dai livelli di 
competenza di docenti e dirigenti, in primis, nell'analizzare e interpretare i 
dati, nell'individuare criticità e problemi nelle performance della scuola e 
nel definire le azioni da intraprendere (assessment literacy). 

A partire dalla considerazione che i dati, a livello macro e micro, con- 
sentano diversi usi, il presente paper riporta la prima fase di un progetto 
di ricerca realizzato in una scuola primaria del Sud Italia e teso a imple- 
mentare un sistema di interim assessment. L'analisi micro dei dati INVALSI 
(anno scolastico 2018/19) ha cosi portato a definire due diversi percorsi di 
sviluppo professionale per gli insegnanti in modo che potessero analizzare 
i dati INVALSI e, comprendere come progettare nuove prove, raccogliere, 
usare e riportare i dati sull'apprendimento degli studenti. Nel richiamare 
gli aspetti principali di questo studio, il contributo riflette criticamente sulle 
modalità con cui veicolare e facilitare un uso estensivo dei dati ricavati dalle 
valutazioni standardizzate ai fini dell'insegnamento. 
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1. Introduction 


The changes undergone by national educational systems in replying to the 
accountability requirements, over the years, have led school personnel (first 
of all, principals and teachers) to face with different evaluation data, infor- 
mation, and methods. The new evaluation and assessment modalities have 
been often perceived as a mere control not really responsive to school reali- 
ties and to instructional needs (DeLuca, 2012; Freddano and Pastore, 2018; 
Stiggins, 2017). The attempts to reduce these, sometimes, radical positions, 
have showed how data, especially those gathered through large-scale assess- 
ment, may be used to support and improve student learning levels (Wiliam, 
2010; 2011). 

The data use to inform teaching action is not new in the educational as- 
sessment field. Teachers have always used student tasks and test results to 
judge how, and what, their students have understood or achieved. What is 
new is the link to accountability systems. For the sake of the truth, if on the 
one hand educational research in this field has demonstrated how large-scale 
assessment data may be useful for the design of improvement actions within 
the school organization, on the other hand, it is clear how teachers and stu- 
dents tend to consider these data not really useful (Castoldi, 2014; Pastore 
and Freddano, 2017; Rogosa, 2005; Supovitz, 2009). The attempts to miti- 
gate the negative effects related to the accountability requirements (e.g. in 
the U.S. context or in Australia and New Zealand), trying to combine large- 
scale and classroom assessments in order to define a more coherent national 
assessment system, have led to new, mixed, assessment modalities, such as 
the interim assessment. Defined as a recurrent assessment realized different 
times over the school year, this assessment measures students’ knowledge 
and skills in order to define their achievement levels and track their progress- 
es through the national curricular standards (Herman, 2016; Crane, 2008). In 
Italy, this kind of assessment has been suddenly identified, in a more practi- 
cal perspective, as “parallel tests” (literally, “prove parallele"). 

The present paper, on the base of the concepts of assessment and data 
literacy nowadays largely widespread in the educational assessment field, re- 
ports on a research project purposed to implement, over two years, an interim 
assessment system within a primary school in the South of Italy. Moreover, 
this research project is aimed to support, through different rounds of pro- 
fessional development paths, the teachers involved in the school data-team. 
More specifically, the paper in the first section offers a brief literature review 
of the studies focused on the concepts of teacher assessment and data liter- 
acy, decision-making, and interim assessment. The second one, instead, re- 
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ports on the first phase of this two years research project aimed to implement 
an interim assessment system with a strong link to the rationale and structure 
of the INVALSI large-scale assessment and of the INVALSI frames of ref- 
erence (Quadri di riferimento). The third part, finally, offers a reflection on 
the main criticalities and implications for educational research and teacher 
education in the Italian school system. 


2. Background and conceptual framework 
2.1. Assessment and data literacy 


The large spread of the evidence-based and data-driven professional 
model for teachers (Thorpe, 2014) has gone along with the need teachers 
avoid, as much as possible, to take decisions using intuitions or anecdotes. 
In this vein, teachers are called to use data gathered from different sources 
to inform their teaching practice and develop, in more general terms, better 
levels of assessment and data literacy (Popham, 2018). While teacher assess- 
ment literacy, for a long time, has been limited only within the context of the 
classroom, nowadays, it is considered more on the school system perspec- 
tive. This conceptual enlargement is relevant in terms of what teachers are 
expected to do. Assessment literacy includes the capacity of teachers to read, 
interpret, and use data for different aims: 

— the school self-evaluation process; 
— the decision-making; 
— the design and implementation of improvement actions (Mandinach and 

Gummer, 2012; Schildkamp and Portman, 2015). 

The relevance recognized to data and to assessment information has 
led recently to the increasing attempts to define teacher assessment litera- 
cy pointing out the need of teachers to use data in a continuous, effective, 
and ethical way. Therefore, teachers should be able to transform information 
gathered through different modalities, such as those offered by the large- 
scale assessment programs, in a knowledge useful for their decision-making 
(Pastore and Andrade, 2019). The emphasis on data has led, sometimes, to 
conflate the just overlapping concepts of assessment and data literacy. In this 
perspective, the concept of data literacy has to be better defined (Mandinach 
and Gummer, 2012): data literacy represents a set of different knowledge 
and abilities that allow teachers to transform data in information, and infor- 
mation in knowledge functional to action. Therefore, the decision-making 
represents a crucial component. 
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Decision-making is the process through which administrators and teachers 
gather and analyse data to guide instructional practice. Furthermore, the wide- 
spread of the decision-making is inextricably connected to the accountability 
requirements. The assumption under the current educational policies is that 
teachers are called to know how to analyse, interpret, and use data in order to 
take aware decisions on how to improve student learning (Datnow, Park and 
Kennedy-Lewis, 2013). Different studies have showed how an effective data 
use made by principals and teachers should bring on a real school improve- 
ment in terms of learning achieved by students. Data, especially those gathered 
from large-scale assessment programs, beyond their limits, criticalities, and the 
frequent requests of revision, (Chudowsky and Pellegrino, 2003), can be used 
for different aims (e.g., school improvement, instructional practice, account- 
ability). Thus, teachers are expected to use data in order to improve teaching 
and learning. However, after decades of accountability pressures, teachers still 
continue to struggle in using data to inform their practice. Sometimes, the rea- 
son is related to teacher data and assessment illiteracy (DeLuca, 2012). More- 
over, teachers have difficulties in interpreting large-scale assessment data and 
in transferring information to review and improve their teaching action. 

There are two possible explanations for this problem: 

1) data and results of large-scale assessment programs are reported in a way 
that does not match the teacher assessment literacy. Thus, teachers have 
difficulties in interpreting data and in using them in an appropriate way at 
classroom and school level; 

2) data content does not correspond to information needed by teachers. So 
difficulties start to arise when teachers have to transfer data in actions. 

A consistent part of studies focused on the first explanation. These studies 
have constantly pointed out how the improvement of teacher assessment and 
data literacy levels is related to the curriculum of teacher education courses 
and to the quality of professional development paths. Other studies, instead, 
have focused on data presentation and interpretation: the key aspect, in this 
perspective, is to present data in a way that match teacher assessment and 
data literacy levels. For example, K. Schildkamp and C.L. Portman (2015) 
address many variables that can influence teacher decision-making: 

— how data-teams are organized in data use; 

— the training of the data-teams on data use; 

— the characteristics of the data-teams; 

— the levels of data literacy and pedagogical content knowledge of teachers 
involved in the data-teams. 

D. Slavit, T.H. Nelson and A. Deuel (2013) have demonstrated how many 
times teachers don't understand what kind of data they need in order to have 
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useful information about student understanding or student thinking. At the 
same time, teachers have not information sufficient to inform their teaching 
action. Another problem in the data use is related to the incapacity of teach- 
ers to ask the right questions to data. As a consequence, teachers have not a 
“meaningful” dialogue with data; so teachers are not able to transform data 
in useful and usable information. 

L. Kallemeyn (2013) has showed how two routines facilitate teachers’ 
data use: teacher collaboration and teacher inquiry. When these two process- 
es are combined they allow teachers to experience the “data-cycle” (Ackoff, 
1989): teachers can gather, analyse, and create knowledge that can be trans- 
formed in teaching action. However, teachers, sometimes, are not able to 
query the data; sometimes, they use data in very simplistic way, disaligned 
with their teaching action. Following E. Farley-Ripple and J. Buttram (2015) 
teachers manifest a literacy gap in establishing which results have to be used 
to effectively and responsively reply to problems highlighted by data. The 
teacher knowledge and the understanding of different modalities for data 
use are really limited. J.A. Marsh, M. Bertrand and A. Huguet (2015) define, 
therefore, this process as “data-practice divide”, the gap teachers demon- 
strate in having, interpreting, and giving data a meaning in order take instruc- 
tional decisions. It is also true, however, that, if on the one hand, contextual 
variables may influence data use, on the other hand, data characteristics and 
data systems may influence how data are used. A. Datnow, V. Park and B. 
Kennedy-Lewis (2013) suggested to look more carefully at modalities teach- 
ers use to interact with data and reply to data on student learning. In this way, 
teachers’ professional learning needs can be identified, and more responsive 
professional development paths can be, therefore, implemented. 

Given this premise, the present paper reports on a possible model of 
integration between the INVALSI data and a system of interim assessment 
coherently balanced and aligned with the national curriculum standards. 
The next paragraph is devoted to illustrate the main aspects of the interim 
assessment. 


2.2. The interim assessment 


Widely spread in the US context, as a consequence of the No Child Left 
Behind policy instances, the interim assessment, despite its clear account- 
ability rationale, is aimed to support school improvement processes focusing 
on the improvement of student learning performances. Moreover, this as- 
sessment is also related to the improvement of actions teachers implement 
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to pursue a continuous and coherent alignment with national curricular stan- 

dards. Precisely because interim assessment replies both to accountability 

and improvement requirements, this kind of assessment is generally inte- 
grated by professional development paths aimed to train teachers to use, in 

a systemic and continuous way, evidence-based assessment and to take in- 

structional decisions responsive to student learning needs. 

The interim assessment is, in this perspective, a powerful lever for the 
improvement of student learning: teachers can use data on student perfor- 
mances to diagnose and monitor student learning; at the same time, teachers 
can modify, adjust, and differentiate their teaching actions. 

The rationale below is clear: more the teaching action replies, with data 
gathered through several assessments, to student learning needs, more chanc- 
es there are to improve instruction. Paraphrasing P. Goren (2010): if we test 
student learning and have data, then we will improve the overall quality of 
the school system. 

Generally, the interim assessment consists in a set of standardized tests 
very similar to national and international large-scale assessment tests. With a 
strong link to the national curricular standards, the interim assessment, how- 
ever, has a reduced number of test items, and it can be administered several 
times over the school year (e.g., from three to five times) (Babo, Tienken 
and Gencarelli, 2014; Henderson et al., 2007; Pereira and Teiken, 2012; Wil- 
liams et al., 2014). 

M. Perie, S. Marion and B. Gong (2009) underline that interim assessment 
presents some similar characteristics with formative assessments, namely, that 
these types of assessment are conducted more frequently and have a narrower 
content focus than do summative assessments. The researchers point out also 
that interim assessment can be used to pursue three main kinds of aims: 

1) instructional. These aims are related to the data use among teachers to 
modify and adapt curriculum and instruction to promote students’ at- 
tainment of learning goals; 

2) evaluative. In this case, assessment results may be used at the classroom, 
school, and regional levels to examine curricular or instructional effecti- 
veness; 

3) predictive. These purposes deal with score use to make judgments about 
how students may perform on an end-of-grade, summative assessment. 
Interim assessment scores can been used to anticipate student performan- 
ces in the summative assessments (e.g., at the end of a module) or in the 
national large-scale assessment program (in this case, however, it is re- 
quired a good alignment between the interim assessment tests developed 
by teachers and the national curricular standards). 
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In Italy, interim assessment, in the terms and characteristics reported 
above, currently is not widely used. Very often, here, interim assessment has 
been associated only with the teachers’ attempts to develop standardized tests 
administrated to students of the same grades within a school. Therefore, the 
implementation of this assessment has been rhapsodic and no systemic, in the 
hands of schools’ and teachers’ initiatives. Moreover, this assessment, when 
implemented, remains largely focused only on the instructional aims because 
limited to the identification of weak areas in the students’ learning. In this per- 
spective, data on student learning levels allow teachers, in most of the cases, 
only to define remedial teaching actions. However, it has to be noted that the 
interest in analyse and document the effects of interim assessment not only on 
student learning, but also on the improvement of teaching action, as well as 
on the impact of educational policy decisions at school and classroom levels, 
is recent. Achieving a meaningful change implies teachers have, as reported 
above in this chapter, a sufficient level of literacy in the educational assess- 
ment domain: if it is true that the interim assessment helps to identify the im- 
provement areas, then it is responsibility of the school personnel to use assess- 
ment information in the right way to revise and adjust instructional actions. 
In this vein, S. Blanc, J.B. Christan, R. Liu, C. Mitchell et al. (2010) confirm 
that the success in the implementation of a good interim assessment system 
depends by the principal’s and teachers’ knowledge and abilities to transfer 
that system in the daily school life. Accountability measures, a strong curric- 
ular guide, and periodic assessments do not assure the school improvement, 
as well as the improvement of student learning performances. Data can make 
more evident the problems, but then are the actors to be called to solve the 
problems. For this reason, it is necessary to support teachers in refining their 
assessment and data literacy through ad hoc professional development paths. 


3. The research project and the teacher professional development 
path 


3.1. Research aims and questions 


In the follow it is reported the first phase of this two years research project 
focused on promote teachers’ assessment and data literacy. More specifical- 
ly, the focus is on how to read, analyse, and use not only the INVALSI data at 
micro-level (classroom level) but also data gathered through the design and 
the implementation of an interim assessment system. In this way, teachers, 
matching different kind of assessment data, should be able to reply more 
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effectively to the school context requirements and needs. In this case, con- 

sidering the latent teachers assessment illiteracy and their lack of experience 

in the interim assessment, it has been decided to use the data gathered only 
for instructional and evaluative aims. 

The study moves from the following questions: 

— RQI1: Is it possible to analyse the INVALSI data at micro-level? 

— SQ2: What are the practical implications for teachers? 

— SQ2: What are professional needs teachers have related to the asses- 

sment and data literacy domains? 

— RQ2: How to implement a system that allows teachers to compare and 
contrast national large-scale assessment program results with school inte- 
rim assessment results? 

INVALSI data (school year 2018/19) have been analysed at micro-level 
and shared with teachers before the dissemination of the INVALSI yearly 
report to schools. Data have been analysed at classroom and student level. 

The next paragraphs bring back on the first research questions and on the 
results of the INVALSI data micro-analysis. Moreover, a description of how 
the interim assessment has been designed (its rationale) is reported. 


3.2. The school context 


The present project involves the “San Giovanni Bosco" school placed in 
two different contexts. In the first case, the school is in the city centre and one 
of the main problems is the population decline; in the second case, instead, 
the school is in a little suburb. Here, the school has to deal with a cultural- 
ly deprived context characterized, however, by the active involvement and 
participation of families in the school life. On the whole, the school has a 
very heterogenic catchment area with low incoming students and a relevant 
number of students with a high socio-economic background. Therefore, the 
school deals with very different problems. For this reason, over the years, 
the educational activities and the remedial initiatives have been purposed to 
reduce gaps between student performances. 

In this perspective, the collaboration between the school and the families 
is a key aspect: the voluntary works of parents and the civic engagement have 
effectively improved the perceptions of internal and external school's stake- 
holders. At the same time, teachers, if on the one hand, have progressively 
oriented their instructional design to a more inclusive perspective, on the oth- 
er hand, they have paid increasing attention to the school efficacy monitoring 
student achievements and reviewing their instructional practices. 
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The present project has been developed, since May 2019, in replying to 
the school’s data-team needs: teachers asked for a conceptual and practical 
knowledge in order to interpret INVALSI results and share them with col- 
leagues for the self-evaluation process. The difficulties to query INVALSI 
data led teachers to ask for different kind of analyses. INVALSI report, in 
fact, while is useful to compare a school with different or similar contexts, 
has been perceived by teachers as not really helpful in reporting information 
about single students or groups of students performances within their school. 
As a consequence, teachers have had several doubts, perplexities, and con- 
cerns about the real use of INVALSI data in their daily teaching practice. 
Moreover, teachers have considered the reporting time of the INVALSI re- 
sults as not functional to pursue the planned improvement actions. Thus, the 
school’s data-team was in trouble with its workload, perceived as unfeasible 
and not useful. 

Once the raw INVALSI data have been gathered (May 2019) a first train- 
ing path has been implemented for teachers in order to support them in the 
analysis and interpretation of results at micro-level. 


3.3. The data 


Gathered micro-data of second and fifth grades have been imported in 
PSPP. First of all, descriptive analyses have been performed: the focus on 
basic statistical analysis has been functional to train teachers to understand 
how to perform data analysis and how to use results on students’ learning. 
Then, the micro-data have been compared with the official INVALSI results. 
These micro-data have been also linked to INVALSI micro-data for each stu- 
dent: thus, using student panels, teachers have been able to compare macro 
and micro-data. More specifically, performing student panels has allowed to 
record “snapshots” of students’ learning. Insofar panels of data are less in- 
formative than event-history data, they have had some benefits for teachers. 
hasten! speaking, in fact, student panels: 

are more informative (more variability, less collinearity, more degrees of 

freedom) and its estimates are more efficient; 

— allow study individual dynamics (e.g., separating age and cohort effects); 

— give information on the time-ordering of events to control for individual 
unobserved heterogeneity, and this benefit is relevant in terms of unob- 
served heterogeneity in non-experimental research. 

As a result, teachers have had very detailed and informative data at school, 
class, and student levels. 
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The following table, for example, reports the basic processing of student 
panels: the normalized dataset (row data) of the fifth-grade students has 
been linked to the INVALSI dataset of the same students when they were 
in the second grade (Tab. 1). The scores have been calculated without con- 
sidering deflective effects related to the threshold value of correct answers 
in multiple tests. 


Tab. 1 — Mean scores (students and ESCS values). Student panels ss.yy. 2015/16 and 
2018/19 


Student ESCS Student ESCS 

S.y. 2015/16 Sy. 2018/19 
High Medium-Medium- Low High Medium-Medium- Low 

High Low High Low 
Italian (2°) 63.07 53.85 61.79 51.55 60.98 63.09 46.10 57.72 
Italian (5°) 72.76 62.57 60.36 57.39 70.28 67.16 54.78 64.68 
Maths (2°) 50.76 | 44.92 4489 47.43 4243 51.60 49.05 48.00 
Maths (5°) 63.37 56.41 58.52 49.70 59.52 63.85 4533 57.41 


English Reatding (2°) 21.70 21.92 20.56 22.61 21.82 22.00 21.75 21.88 
English Reading (5°) 771.00 67.52 87.04 70.37 70.55 79.26 70.76 74.34 
English Listening (5°) 66.12 54.13. 71.30 51.18 58.73 69.75 45.22 59.79 


While teachers, following this micro-analysis process have had detailed, 
usable, and robust information on students’ performances, it has to be noted 
that the process of normalization of the second and fifth grades data was 
really expansive in terms of time and workload (e.g., elaboration and test of 
code strings). The result has been a good level of information but not timely 
for teachers in order to review their instructional practice and to design plans 
and strategies for the school improvement. 


4. Discussion and next steps 


Even though the inquiry path has showed how it is possible to analyse the 
INVALSI large-scale assessment data at micro-level (RQ1), it has been sud- 
denly evident how complex were the practical aspects and the implications 
for teachers (SQ1). Moreover, this first phase has been useful to show how 
deep were the teachers' gaps in the assessment and data literacy field (SQ2). 
Therefore, the second phase of the research project has been devoted to sup- 
port teachers in the implementation of an interim assessment system. This 
system should allow teachers to gather data on students' learning in a more 
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sustainable and smart way, and then compare these data with the INVALSI 
results. Thus, a new professional development path has been realized in or- 
der to train teachers to design and construct standardized tests. 

More specifically, teachers have been train on how: 

— to establish the main aims of the tests; 

— to examine the clarity and the specificity of aspects and traits measured 
by the tests; 

— to arrange a sufficient number of test items in order to take a valid deci- 
sion; 

— toanalyse test items perceived as more difficult by students (or items with 
more errors or wrong answers); 

— toexamine a sample of items for having information on a wider compe- 
tency domain; 

— to disaggregate data for different groups of students. 

The implementation of the interim assessment system is currently on the 
going and will continue through the school year 2020/21. The interim tests 
will be administered, in two different rounds during the school year, not only 
to the INVALSI target students (second and fifth grades) but also to third and 
fourth grades students. The last aim, in fact, is to define an interim assess- 
ment system that, avoiding inquisitive lectures or obsessive controls, allow 
teachers to reflect on and to review their instructional practices. 


5. Conclusions 


Despite the current diffusion and the relevance of data in the school con- 
text, there are different problems and open questions related to the teacher 
data-driven decision-making and to the assessment data use. Educational 
policies, not only in Italy, sometimes don't support school actors with clear 
explanations on how to use data for the improvement of the school neither 
on how to effectively use data. 

Having data is not a guarantee of the improvement. 

At the school level, data can differently impact decision-making. The 
sense-making theory, in this perspective, has clearly explained how actors 
tend to interpret, adapt, or transform the action directions. The decision-mak- 
ing is not focused only on the data use for instructional activities within the 
classroom. Thus, the most relevant aspect in this process is represented by 
the change of the school culture. 

The professional development path implemented after the micro-anal- 
ysis of the INVALSI results has trained teachers to pursue more clear and 
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tailored learning goals and will support them in interpreting the interim 
assessment data. If a teacher doesn’t know how a test has been designed 
and realized, if he/she doesn’t know how scales (or sub-scales) have been 
used and how the standards have been defined, he/she will never be able to 
interpret a percentage or the rate on student proficiency, he/she will never 
really understand what data mean and imply for students and for teaching 
practice. If assessment results are disaggregated for learning goals it is 
possible to have information that teachers will use more easily in their 
instructional practice. 

Hence the implications related to the use of standardized tests are several. 
Data gathered through large-scale and interim assessment should be useful 
for teachers to: 

— define what are instructional priorities; 

— define further teaching actions for students who have difficulties in lear- 
ning; 

— identify, more easily, student learning strengths and design instructional 
actions that support them in fostering their leaning; 

— review teaching methods and strategies; 

— establish, at the micro-level of the classroom as well as at macro-level 
of the school system, if and how adjust the instructional curriculum (in 
terms of local student learning goals and expected outcomes) to the infor- 
mation gathered about student learning (Stiggins, 2017). 

There are relevant and different implications in terms of educational 
policies and practices because how teachers use data, and more specifically 
assessment data, to support decision-making and instruction, is not only a 
“technical” question. Moreover, interpreting, and using data gathered from 
an interim assessment system constrain teachers to act in a more function- 
al and responsive way to the local context needs but also to the national 
context requirements (Little, 2012). Beyond the classrooms, teachers have, 
therefore, the chance to really identify and manage the factors that impact on 
the alignment between the national evaluation school system and their daily 
teaching practice. In this perspective, there should be more chances to under- 
stand if teachers’ assessment practices and their data-driven decision-making 
are responsive to innovations and to educational, social, and institutional 
transformations (Levin and Datnow, 2012). 


93 


ISBN 9788835131076 


References 


Ackoff R.L. (1989), “From data to wisdom”, Journal of Applied Systems Analysis, 
16, pp. 3-9. 

Babo G., Tienken C.H., Gencarelli M.A. (2014), “Interim testing, socio-economic 
status, and the odds of passing Grade 8 state tests in New Jersey”, Research in 
Middle Level Education, 38, 3, pp. 1-9. 

Blanc S., Christman J.B., Liu R., Mitchell C., Travers E., Bulkley K.E. (2010), 
“Learning to learn from data: Benchmarks and instructional communities”, Pea- 
body Journal of Education, 85, 2, pp. 205-225. 

Castoldi M. (2014), Capire le prove INVALSI, Carocci, Roma. 

Chudowsky N., Pellegrino J.W. (2003), “Large-scale assessments that support learn- 
ing: What will it take?”, Theory Into Practice, 42, 1, pp. 75-83. 

Crane E. (2008), Interim assessments practices and avenues for state involvement, 
Council of Chief State School Officers, Washington (DC). 

Datnow A., Park V., Kennedy-Lewis B. (2013), *Affordances and constraints in the 
context of teacher collaboration for the purpose of data use", Journal of Educa- 
tional Administration, 51, 3, pp. 341-362. 

DeLuca C. (2012), “Preparing teachers for the age of accountability: Toward a 
framework for assessment education", Action in Teacher Education, 34, 5/6, pp. 
576-591. 

Farley-Rippl E., Buttram J. (2015), “The development of capacity for data use: The 
role of teacher networks in an elementary school", Teachers College Record, 11, 
4, pp.1-34. 

Freddano M., Pastore S. (a cura di) (2018), Per una valutazione delle scuole oltre l'adem- 
pimento. Riflessioni e pratiche sui processi valutativi, FrancoAngeli, Milano. 

Goren P. (2012), *The practice of data use: Data, data, and more data — What's an 
educator to do?", American Journal of Education, 118, 2, pp. 233-237. 

Henderson S., Petrosino A., Guckenburg S., Hamilton S. (2007), REL Technical 
Brief-a second follow-up year for “Measuring how benchmark assessments 
affect student achievement", Department of Education, Institute of Education 
Sciences, National Center for Education Evaluation and Regional Assistance, 
Regional Educational Laboratory Northeast and Islands, Washington, DC. 

Herman J. (2016), Comprehensive Standards Based-Assessment Systems Supporting 
Learning, National Center for Research on Evaluation, Standards, and Student 
Testing (CRESST), Los Angeles. 

Kallemeyn L. (2014), “School-level organizational routines for learning: Supporting 
data use", Journal of Educational Administration, 52, 4, pp. 529-548. 

Levin J.A., Datnow A. (2012), “The principal role in data-driven decision making: 
Using case-study data to develop multi-mediator models of educational reform", 
School Effectiveness and School Improvement, 23, 2, pp. 179-202. 

Little J.W. (2012), “Understanding data use practice among teachers: The contri- 
bution of micro-process studies", American Journal of Education, 118, 2, pp. 
143-166. 


94 


ISBN 9788835131076 


Mandinach E.B., Gummer E.S. (2012), Navigating the landscape of data literacy: 
It is complex, WestEd/Education Northwest, Washington (DC)/Portland (OR). 

Marsh J.A., Bertrand M., Huguet A. (2015), “Using data to alter instructional prac- 
tice: The mediating role of coaches and professional learning communities”, 
Teachers College Record, 117, 4, pp. 1-41. 

Pastore S., Andrade H.L. (2019), “Teacher assessment literacy: A three-dimensional 
model”, Teacher and Teaching Education, 86, pp. 128-138. 

Pastore S., Freddano M. (2017), “‘Questione di feedback’: dati INVALSI e pratiche 
di valutazione in classe", in P. Falzetti (a cura di), L'uso dei dati INVALSI, Fran- 
coAngeli, Milano. 

Pereira M., Tienken C. (2012), *An evaluation of the influence of interim assess- 
ments on Grade 8 student achievement in mathematics and language arts", /nter- 
national Journal of Educational Leadership Preparation, 7, 3, pp. 1-13. 

Perie M., Marion S., Gong B. (2009), *Moving toward a comprehensive assessment 
system: A framework for considering interim assessment", Educational Measu- 
rement: Issues & Practice, 28, pp. 5-13. 

Popham W.J. (2018), Assessment literacy for educators in a hurry, ASCD, Alexan- 
dria (VA). 

Rogosa D. (2005), “Statistical misunderstandings of the properties of school scores 
and school accountability", in J.L. Herman, E.H. Haertel (eds.), Uses and misus- 
es of data for educational accountability and improvement, Blackwell Publish- 
ing, Malden (MA). 

Schildkamp K., Lai M.K., Earl L. (eds.) (2013), Data-based decision making in ed- 
ucation: Challenges and opportunities, Springer, Dordrecht (NL). 

Schildkamp K., Portman C.L. (2015), “Factors influencing the functioning of data 
teams”, Teacher College Record, 117, 4, pp. 1-43. 

Slavit D., Nelson T.H., Deuel A. (2013), “Teacher groups’ conceptions and uses of 
student learning data”, Journal of Teacher Education, 64, 1, pp. 8-21. 

Stiggins R.J. (2017), The perfect assessment system, ASCD, Alexandra (VA). 

Supovitz J. (2009), “Can high stakes testing leverage educational improvement? 
Prospects from the last decade of testing and accountability reform", Journal of 
Educational Change, 10, 2/3, pp. 1-227. 

Thorpe R. (2014), “Sustaining the teaching profession”, New England Journal of 
Public Policy, 2, 1, pp. 1-16. 

Wiliam D. (2010), “An integrative summary of the research literature and implica- 
tions for a new theory of formative assessment”, in H.L. Andrade, G.C. Cizek 
(eds.), Handbook of formative assessment, Routledge, New York. 

Wiliam D. (2011), Embedded formative assessment, Solution Tree Press, Bloom- 
ington (IN). 

Williams R.T., Swanlund A., Miller S., Konstantopoulos S., Eno J., van der Ploeg 
A., Meyers C. (2014), “Measuring instructional differentiation in a large-scale 
experiment", Educational and Psychological Measurement, 74, pp. 263-279. 


95 


ISBN 9788835131076 


6. Links between the INVALSI Mathematics test 
and teaching practices: an exploratory study 


by Ferdinando Arzarello, Federica Ferretti 


The chapter shows the first results of an interdisciplinary project aimed at 
investigating the link between the Mathematical IN VALSI tests and the teach- 
ing and learning processes of Mathematics, in particular with didactic practic- 
es. The research project is conducted by the INVALSI Group - Disciplinary 
Didactics of the SIRD — Italian Didactic Research Society. The group is com- 
posed by disciplinary experts (from the Universities of Turin, the Free Uni- 
versity of Bolzano, the University of Milan and the University of Bari) and 
educationalist (from the University of Bologna, the University of Turin and 
the Sapienza University of Rome) coordinated by Prof. Arzarello and Prof. 
Vannini. The interdisciplinary study consisted in the construction of a tool for 
detecting teachers' attitudes towards INVALSI, both towards the INVALSI as 
an Institute, with its aims and working methods, and towards the INVALSI 
Mathematical tests and their effects on teaching. The aim is to understand 
which are the “tools” that the teachers have and, above all, use to read and in- 
terpret the INVALSI standardized assessment and which “tools” are available 
to identify possible effects of INVALSI tests on Mathematics teaching. The 
aim of the research is to identify professional development teachers’ needs at 
national level within schools and to propose guidelines for improving practic- 
es regarding the use of INVALSI surveys. To answer the research hypotheses, 
a questionnaire was designed and administered to investigate teachers' beliefs 
regarding the knowledge and skills investigated by the INVALSI tests, their 
closeness to teaching practices in Mathematics and the role that they assume 
within the context school. In detail, the questionnaire consists of two areas of 
variables, one specifically for Mathematics Education and one relating to the 
aspects of Education. In detail Mathematics Education variables are aimed 
at investigating how much the Mathematical contents and abilities detected 
with the INVALSI tests are — more or less close to daily personal teaching 
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practices — lived as coherent/inconsistent with the national guidelines and are 
recognized or not consistent with the intentions of INVALSI — considered 
useful for influencing/innovating personal teaching practice. Questions about 
teachers’ ability to read INVALSI data were in the questionnaire; these vari- 
ables represent the focus of the survey and they are the dependent variables. 
The independent variables are those related to the aspects of Education and 
Didactics that refer to constructs aimed at detecting the attitudes of teachers 
and all scholastic subjects. The research that is taking place is a correlation 
type; the first data collected are being analyzed. 


Il capitolo mostra i primi risultati di un progetto interdisciplinare volto 
ad indagare il legame tra le prove INVALSI di Matematica con i processi 
di insegnamento e apprendimento della Matematica, in particolare con le 
pratiche didattiche. Il progetto di ricerca é condotto dal Gruppo INVALSI 
-Didattiche Disciplinari della SIRD — Società Italiana di Ricerca Didattica, 
formato da esperti disciplinaristi (appartenenti alle Università di Torino, 
alla Libera Università di Bolzano, all’Università Statale di Milano e all'U- 
niversità di Bari) e pedagogisti (appartenenti all’Università di Bologna, al- 
l'Università di Torino e alla Sapienza Università di Roma) coordinato dal 
prof. Arzarello e dalla prof.ssa Vannini. Il lavoro interdisciplinare è consi- 
stito nella costruzione di uno strumento di rilevazione degli atteggiamenti 
degli insegnanti nei confronti dell’INVALSI, sia nell’INVALSI come Isti- 
tuto, con le sue finalità e modalità di lavoro, sia nei confronti delle prove 
INVALSI di Matematica e delle loro ricadute sulla didattica. Lo scopo è 
quello di comprendere quali sono gli “strumenti” che gli insegnanti hanno 
e, soprattutto, utilizzano per leggere e interpretare le rilevazioni INVALSI 
e di quali “strumenti” dispongono per individuare possibili ricadute delle 
prove sulla didattica della Matematica. L’obiettivo della ricerca è quello di 
identificare i bisogni formativi a livello nazionale all’interno delle scuole e 
proporre linee guida per il miglioramento delle prassi per quanto riguarda 
l’uso delle rilevazioni INVALSI. Per rispondere alle ipotesi di ricerca è stato 
progettato e somministrato un questionario volto a indagare le convinzioni 
degli insegnanti per quanto riguarda le conoscenze e le competenze indagate 
dalle prove INVALSI, la loro vicinanza alle pratiche didattiche in Matema- 
tica e ruolo che assumono all’interno del contesto scolastico. In dettaglio, 
il questionario è costituito da due ambiti d variabili, uno specificamente di 
Didattica della Matematica e uno relativo agli aspetti di Didattica gene- 
rale. In dettaglio variabili di Didattica della Matematica sono finalizzate 
a rilevare quanto i contenuti e le abilità matematiche rilevate con le prove 
INVALSI siano: a) più o meno vicini alle pratiche didattiche personali quo- 
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tidiane; b) vissute come coerenti/incoerenti con le Indicazioni nazionali e 
siano riconosciute o meno in modo coerente con le intenzioni di INVALST; 
c) ritenute utili per influenzare/innovare la pratica didattica personale. Sono 
state inoltre proposte domande sulla capacità di lettura dei dati INVALSI da 
parte degli insegnanti; queste variabili rappresentano il focus dell'indagine 
e sono le variabili dipendenti. Le variabili indipendenti sono quelle relative 
agli aspetti di Didattica generale che fanno riferimento a costrutti volti alla 
rilevazione degli atteggiamenti di insegnanti e di tutti i soggetti scolastici. 
L'indagine che si sta svolgendo è di tipo correlazione; i primi dati raccolti 
sono in fase di analisi. 


1. Rationale 


This chapter shows the first results of an interdisciplinary research pro- 
ject aimed at investigating the links between INVALSI tests of Mathematics 
and Mathematics teaching and learning processes, especially with respect to 
teachers’ didactic practices. The research project is conducted by the “Gruppo 
INVALSI — Didattica e Saperi Disciplinari” of the SIRD (Italian Society of 
Didactic Research), which includes experts in Mathematics and in Pedagogy. 
The coordinators of the group are Ferdinando Arzarello (Mathematics: Uni- 
versity of Turin) and Ira Vannini (Pedagogy: University of Bologna)!. The 
aim ofthe study is to investigate Mathematics teachers' beliefs about INVALSI 
surveys, in particular about the way they read and interpret IN VALSI surveys 
data, and in what measure INVALSI tests effectively impact on their Mathe- 
matics teaching practices. 

In line with the goals of the SIRD, a broader objective of our research is to 
identify training needs at national level within schools, and to propose guide- 
lines for the improvement of practices regarding the use of INVALSI tests. 
In details, our study is part of a broader line of research aimed at improving 
a close link between standardized assessments and Mathematics education. 
Its aim 1s to find an effective way to merge standardized assessments' results, 
methods, theoretical frameworks and tools — that are designed in order to 
impact at a systemic level — into actions of teachers and schools (Doig, 2006; 


! The researchers of the project are: Barbara Balconi (University of Milan), Giorgio Bo- 
londi (Free University of Bozen-Bolzano), Eleonora Faggiano (University of Bari), Federi- 
ca Ferretti (University of Ferrara), Violetta Lonati (University of Milan), Daniela Maccario 
(University of Turin), Annarita Monaco (Teacher, Rome), Ottavio Rizzo (University of Mi- 
lan), Roberto Trinchero (University of Turin), Valentina Vaccaro (INVALSI, Rome). 
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Looney, 2011). In order to fully acknowledge the potentials and educational 
aims of standardized assessment we need effective theoretical tools to inter- 
pret the quantitative data they provide and the macro-phenomena that emerge 
from the complexity of educational systems. The use of the standardized as- 
sessment can truly improve the teaching and learning of Mathematics only if 
it is able to give refined, culturally wide-ranging and operational information 
to policy makers, teacher training programs, curriculum developers, princi- 
pals and teachers (De Lange, 2007). Our research moves exactly within this 
stream of thought: in particular, its first part investigates teachers’ beliefs on 
standardized assessment and tools and on the way they actually use to read 
and interpret standardized tests and data. 


1.1. The INVALSI test and Mathematics education 


As we underlied, our research is part of a study within a broader strand 
of international research regarding the link between Large Scale Assessment 
(LSA) results and Mathematics Education Research (De Lange, 2007) and, 
in particular, the central role that analysis of standardized assessment data 
may have for teachers’ professional development. 

Specifically, INVALSI provides annually the data results, based on a sta- 
tistically significant national sample, for every single item of all INVALSI 
tests and in our research these data have been used. The framework adopt- 
ed by INVALSI assessment tests (INVALSI, 2018) is strictly connected to 
the Italian National Guidelines, includes aspects of mathematical modelling 
adopted in PISA research, and is developed according to results provided by 
Mathematics education research. These facts show the link between INVALSI 
tests and results from research in Mathematics education. The INVALSI pub- 
lishes the results of the national sample of each item and gives back to the 
schools the results relative to each grade; the results are issued annually and 
can provide important information for categorizing students’ errors at a macro 
level. As already shown in various research studies (e.g. Ferretti and Bolondi, 
2019), the results of the INVALSI surveys highlight didactic macro-phenom- 
ena that can provide very useful information on learning/teaching processes. 
As we will see in the examples presented below, one of the focuses of our in- 
vestigation is to study if and how teachers are aware and properly understand 
the macro-phenomena emerging from INVALSI assessments. 
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2. The research 


As hinted above, the general aim of our research is to identify training 
needs at a national level and to propose guidelines for the improvement of 
Mathematics practices with the use of INVALSI tests. The general objectives 
of our study are: 

— to investigate teachers’ beliefs regarding the knowledge and skills investi- 
gated by the INVALSI tests; 

— to investigate the relation between INVALSI Mathematics tests and Ma- 
thematics teaching and learning processes in the classroom, in particular 
about the didactic practices adopted by teachers. 

— More specifically, the aim is to understand: 

— (level 1) which tools teachers actually use to read and interpret INVALSI 
tests and data; 

— (level 2) which means are available to researchers in order to identify 
the possible effects of INVALSI tests on mathematics education in our 
schools. 

To answer the research questions, a questionnaire was designed and ad- 
ministered. Before the final administration to a large sample of Italian teach- 
ers, a try-out was performed; some of the results of the try-out are presented 
in the following paragraphs. The questionnaire aimed at investigating as- 
pects such as teachers’ awareness of the learning objectives detected by the 
INVALSI tests, their conceptions of errors in Mathematics, their use of tests 
in daily teaching, their misconceptions about standardized tests, their idea of 
assessment and, in particular, of formative assessment. It is built basing on 
variables from two areas: one specifically for Mathematics Education and 
one relating to aspects of General Education. The questions were prepared 
considering teachers’ ability to read INVALSI Mathematics data as well 
as using constructs aimed at detecting more general teachers’ attitudes and 
practices. The interdisciplinary work led to the construction of a tool apt to 
detect the attitudes of teachers towards INVALSI in two main directions: 
both of INVALSI as an Institute with its aims and methods of work, and of 
INVALSI Mathematics tests and their impact on teaching practices. 

Mathematics Education variables are aimed at detecting to what extent 
the Mathematical contents and abilities identified within INVALSI tests are: 
— more or less “close” to daily personal teaching practices; 

— experienced as consistent/inconsistent with the National Guidelines and 
whether they are recognized as consistent or not with the intentions of 
INVALSI; 

— deemed useful for influencing/innovating personal teaching practices. 
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The try-out of the questionnaire consisted in its administration to 85 
primary school Mathematics teachers. This first part of the analysis mainly 
investigates the types of correlation between the involved variables. In the 
following section we will present some highlights of the try-out of the ques- 
tionnaire, providing examples from the section of Mathematics education. 


3. Examples 


This paragraph illustrates three examples of reflections inherent to 3 
INVALSI Mathematics items, which emerged after the administration of 
the questionnaire. In addition to the questions focused on 7 items of the 
INVALSI tests which highlight didactic macro phenomena, at the end of the 
Mathematics education section there are some transversal questions. Two 
of these questions ask how suitable INVALSI items are for assessing students 
learning and how commonly they are used in assessment practices. The fol- 
lowing analyses will also illustrate how the investigated items position with 
respect to these two questions. 


3.1. Decimals 


The first of the questions investigated is related to the following item, 
administered in the Mathematics INVALSI test for grade 5 Italian students 
in the s.y. 2008/09. 


10. To which number do they correspond "12 tens, 7 tenths and 2 thousandths"? 
O A 12,702. 
[] B. 120,702. 
[i € 12772 
G iD 120,72. 


Fig. 1 — Item 10, grade 5 Mathematics INVALSI test 2009 


Teachers are asked to estimate the degree of difficulty of the question, 
in terms of “how difficult do they find this question at the end of the fifth 
grade". 
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The following figure (Fig. 2) shows the results with reference to 82 col- 
lected responses (facilissima = very easy, difficilissima = very difficult). 


1 (Facilissima) 


1 (Facilissima) 9 9 5.03% 5.03% 10.98% 10.98% 
2 16 25 8.94% 13.97% 19.51% 30.49% 
3 12 37 6.7% 20.67% 14.63% 45.12% 
4 9 46 5.03% 25.7% 10.98% 56.1% 
5 11 57 6.15% 31.84% 13.41% 69.51% 
6 7 64 3.91% 35.75% 8.54% 78.05% 
7 8 72 447% 40.22% 9.76% 87.8% 
8 8 80 4.47% 44.69% 9.76% 97.56% 
9 1 81 0.56% 45.25% 1.22% 98.78% 
10 (Difficilissima) 1 82 0.56% 45.81% 1.22% 100% 
Sum: 82 : 45.81% : 100% : 

Not answered: 97 . 54.19% . 

Average: 4.26 Minimum: 1 Variance: 5.58 

Median: 4 Maximum: 10 Std. deviation: 2.36 


Total answered: 82 


Fig. 2 — The results in reference to the "Decimal" question 


The question requires a conversion transformation between two different 
semiotic registers (Duval, 2006) and the low percentage of correct answers 
at national level (33%) highlights that managing this conversion is difficult 
for many Italian students. The perception of the difficulty of the question by 
teachers is very different from the national trend: looking at the cumulative 
probability we note that almost 70% answered “5”. The data suggest a fail- 
ure to perceive the degree of difficulty of a question. Reflections inherent 
in this item suggest that it could be very interesting also to investigate this 
non-correspondence between a perceived difficulty and a national difficulty 
also in other questions. In the try-out it is the only item in which we do not 
inform about the percentages of correct answers. This fake perception of the 
difficulty level of the item poses also an interesting question to investigate: it 
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may be indeed significant to scrutinize whether, and to what extent, this fake 
perception is connected with the fact that the item is one of those considered 
“most suitable for assessing learning” and one of the “most commonly used 
in assessment tests". 


3.2. Ribbon bow 


Other interesting results emerged from the following INVALSI question, 
administered to students of the fifth primary class in the s.y. 2008/09. 


17. Alessandra buys a book at the supermarket; at home she prepares a 
package similar to this: 


5cm—» 


15cm 


How many cm of ribbon did she use in all, knowing that 30 cm was 
needed to make the bow? 


[] A. 4L 
L) B. T 
Gi GO. iz. 
O D. 122 


Fig. 3 — Item 17, grade 5 Mathematics INVALSI test 2009 


In the questionnaire, the percentage score for correct responses at nation- 
al level (14.796) was given. 

Various possible causes of students' errors were then reported and teach- 
ers were asked to indicate which, by their opinion, was the main cause of 
students’ difficulties. 

The sentences were: 

— because the students don't have sufficient spatial visualization skills; 
— because the students didn't do enough manipulative activities; 

— because the students didn't understand the task; 

— because the students made wrong calculations; 

— because the students didn't read the text carefully; 
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— because the students thought only about performing calculations; 
— other (specify). 

The graphs in the follow figure (Fig. 4), show the percentage of choice 
for each sentence. 


because the students don't have 
sufficient spatial visualization skills; 


because the students didn't do enough 
manipulative activities; 


because the students didn't understand 
the task; 


because the students made wrong 
calculations; 


because the students didn't read the text 
carefully; 


because the students thought only about 


performing calculations; 
other (specify) 
0 3 6 9 12 15 18 21 24 27 30 33 36 39 43 
Frequency table 
tense the students don't have sufficient spatial visualization 43 43 41.75% 24.02% 24.02% 58.9% 58.9% 
because the students didn't do enough manipulative activities; 24 67 23.3% 13.41% 37.43% 32.88% 91.78% 
because the students didn't understand the task; 4 71 3.88% 2.23% 39.66% 5.48% 97.26% 
because the students made wrong calculations; 5 76 4.85% 2.79% 42.46% 6.85% 104.11% 
because Sho siodente dERT read Do Mit cale 12 88 11.65% 6.7% 49.16% 16.44% 120.55% 
aaa a aa 8 96 7.77% 447% 5363% 10.96% 131.5196 
other (specify) 7 o3 6.8% 3.91% 57.54% 9.59% 141.1% 
Sum: 103 - 100% - - - - 
Not answered: 106 : - 59.22% - - 
Average: 2.72 Minimum: 1 Variance: 4.17 
Median: 2 Maximum: 7 Std. deviation: 2.04 


Total answered: 73 


Fig. 4 — The results in reference to the "Ribbon Bow” question 


Almost 60% of teachers attribute the low percentage of correct answers 
at national level to the fact that students do not have sufficient spatial visual- 
ization capacity and 33% to the fact that students did not do enough manip- 
ulative activities. 

At national level, the correct answer (D) was selected by 14.7% (one of 
the worst performances in closed-ended answers since 2008) while option A 
was chosen by 28.9% and option B by 41.5%. 

Option A shows the number 41 which corresponds exactly to the sum of 
the numbers shown in the figure (5+15+21) while option B (71) corresponds 
to the sum of the numbers in the figure and the only number expressed in 
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figures in the text (41+30). The causes of the difficulties, framed by the di- 
dactic contract construct in the sense of Brousseau (1988), are quite evident 
and they do not correspond or correspond only partially to those recognized 
by teachers. We are therefore faced with a lack of awareness of the causes 
of the error. It will certainly be interesting to consider this issue in depth, to 
try to investigate whether the causes of these interpretative difficulties are 
linked to difficulties in interpreting students’ difficulties or to difficulties in 
understanding the goal of the question. 

Regarding the transversal questions, this item is considered by teachers 
as one of the least “suitable” for assessing learning and one of the questions 
“least used in assessment tests”. 


3.3. Cotton balls 


Relevant results are those inherent to the question concerning the follow- 
ing item, given in the INVALSI grade 5 test in the s.y. 2012/13. 


D11. To make 4 crochet tablecloths the grandmother uses 6 cotton balls. 


a. How many balls of the same type must she use to prepare 20 
tablecloths? 
jars p €—— M 

b. Write how you found the answer. 


TI III Ida UIL 


Fig. 5 — Item 11, grade 5 Mathematics INVALSI test 2013 


The following table shows some answers from grade 5 students (Primary 
School). Some possible answers plausibly provided by the students (some 
taken from Arzarello, 2018) and others designed by the group of researchers 
were then proposed. The teachers were asked to indicate how they would 
rate them. The following table shows the answers in reference to each pro- 
posed sentence (Completamente corretta = Completely correct, Parzialmente 
corretta — Partially correct, Prevalentemente scorretta — Mainly incorrect, 
Completamente scorretta — Completely incorrect). 
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Tab. 1 — Results of each sentence — Cotton balls item 


Parzialmente corretta 


a. “First, I calculated 6-4 = 2; after prevalentemente scorretta 
the 20, I added 2 and I got 22” 


Completamente scorretta 


Completamente corretta. 


b. “I multiplied 6 balls for 4 place- aes 
mats and I got 24” Prevalentemente scorretta 


Completamente scorretta 


0 4 68 12 16 20 24 28 32 36 40 44 48 Sì 


Completamente corretta 


Parzialmente corretta. 


c. “I have multiplied 6 balls by 20, 
for 20 placemats” Prevalentemente scorretta 


Completamente scorretta 


0 4 8 12 16 20 24 28 32 36 40 44 48 51 


d. “Since, for 4 placemats, we need 
6 balls of yarn (so 2 more), for 20 
place mats I just add 10” 


Parzialmente corretta 


Prevalentemente scorretta 


Completamente scorretta 


Completamente corretta 
e. “The grandmother uses a ball and Parzialmente corretta 
a half to prepare a placemat so I did 
this = 20 x 1.5 = 30 (in one corner 
of the sheet is the multiplication in a 
column)” 


Prevalentemente scorretta 


Completamente scorretta 


Completamente corretta 
Parzialmente corretta 
f. “Because I calculated 20+6+4” 


Prevalentemente scorretta 


Completamente scorretta 


One of the most interesting data is certainly the “noise” in the answers 
inherent to our solution d, in which the strategy inherent to the correct an- 
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swer uses the mixed addition-multiplication model and does not explain all 
the steps. 

In fact, while the teachers evaluate incorrect the first three answers (as 
they in fact are), there is a lot of confusion as far as the fourth is concerned. 
Why do almost 40% of teachers not accept d as correct? Because not all 
the steps are made explicit or because they do not recognize it as a possible 
correct strategy? 

Finally, the last option is the only one for which there is low coherence in 
answers to the two final questions that we have always asked in each item. 
A consequent suggestion for changes in the text is to modify it, maintaining 
the combination right answer-wrong strategy. In fact, together with item 2 
(ribbon bow), this item is considered by teachers to be the least “suitable for 
assessing learning” and one of the questions “least used in assessment tests". 


4. Discussion 


In the paper we have presented the first findings from the try-out of a 
questionnaire aimed at investigating the way primary school Mathematics 
teachers read and interpret INVALSI surveys data. 

To discuss what we have found until now, we make an analogy between 
the small context of an usual mathematics classroom and the general na- 
tional school context, which our research is considering. In the analysis of 
classroom interactions it is common to see what Anna Sfard calls an “incom- 
mensurable discourse" (Sfard, 2008) between the teacher and the students: 
they use the same words but with a different meaning and moreover are not 
aware of the difference; a conflict is generated, which, if not overcome, can 
have serious consequences for successful teaching/learning processes in the 
classroom. 

Something similar happened to us when we analyzed the answers to our 
questionnaire: also here we found incommensurable languages. They are the 
sign of what we call a three-fold meta-didactical conflict: it has been possible 
to realize its existence thanks to the questionnaire. As its name suggests, this 
conflict has three components and is meta-didactic since it concerns discours- 
es about didactic processes like assessment, students’ competencies and mis- 
takes, etc., and not about the thought mathematics concepts themselves, as it 
Is in the case of usual epistemic or didactic conflicts reported in the literature 
(e.g. by Brousseau, 2002 or Sfard, 2008). We will now sketchily describe it. 

A first component of the conflict concerns the fact that many teachers per- 
ceive the difficulty of a question from the INVALSI survey very differently 
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from the national trend (see example 3.1). A second component concerns 
the fact that many teachers interpret the difficulties of students (see example 
3.2) or evaluate their answers (see example 3.3) to the INVALSI tests in a 
way that is completely different from what unquestionably appears from the 
data of the survey. A third component is a consequence of the previous two 
and concerns the contradictory way according to which teachers interpret the 
rationale of the INVALSI tests (for example, see how they couple the dyads 
suitable/not suitable Vs most used/not used in the examples with respect to 
what appears in the survey data). 

Of course, it is possible that the three components may be only the epi- 
phenomenon of a deeper conflict, whose nature at the moment we have not 
yet understood, but until now we can speak only of a three-fold conflict, 
since its three components in any case appear deeply intertwined. 

We think that our current hypothesis about this conflict will be confirmed 
or refuted through the administration of the questionnaire to a wider court of 
subjects and a deep analysis of the related data. In case of confirmation, it 
will be possible to refine the same analysis of the conflict, deciding about its 
three-fold or different nature, and also clarifying its deep structure and nature, 
e.g. with respect the knowledge and beliefs of teachers. Basing on the anal- 
ogy from the conflicts in the classroom, for which a successful strategy for 
overcoming them is generated by a clear understanding of their nature, it will 
be precisely from a clean picture of the structure and dynamics of our conflict 
that it will be possible to design suitable guidelines for getting rid of it and 
obtaining a real improvement of practices regarding the use of INVALSI tests 
in the school. But this will be possible only with a further step of our research, 
after the completion of the current one with the analysis of a suitable number 
of subjects. 
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7. The INVALSI data for the 2030 Agenda 


by Barbara Baldazzi 


The 2030 Agenda for sustainable development adopted from the UN-As- 
sembly General (UN Resolution A7RES/70/1, New York) is built on 17 Sus- 
tainable Development Goals (SDGs) with the aim to end poverty, protect the 
planet and ensure prosperity for all by 2030. The 17 goals are articulated in 
169 targets and 244 indicators (232 of which are different) for monitoring of 
the Goals. 

Goal 4 monitoring “Quality education for all” for our country focuses on 
the training path of people from access to learning activities at five years, 
continuing with inclusion in secondary and tertiary education levels, moni- 
toring skills and learned knowledge. Quality education and lifelong learning 
opportunities for all are central to ensuring a full and productive life to all 
individuals and to the realization of sustainable development. Targets to be 
monitored concern different dimensions: access for all to education of all 
levels (primary, secondary and tertiary), the quality of education, the posses- 
sion of knowledge and skills for employment and sustainable development; 
the elimination of gender disparities in education and equal access for the 
most vulnerable; monitoring of school facilities, so that they are suitable for 
everyone’s needs. 

The present work shows how the integration of data from various sources 
(INVALSI data, data from ISTAT survey and data from MIUR) can give 
a global picture of the situation in Italy; and how from the aggregation of 
data, from the analysis of each single indicator and from the involvement of 
other targets and objectives of the 2030 Agenda, substantial information on 
inequalities can arise. 

Every child and young person has the right to learn, train, develop his 
skills, competences and aspirations in the most profitable possible way and 
with the best opportunities; when this right is not guaranteed, the child finds 
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himself in a condition of educational poverty, suffers from a lack of opportu- 
nity, which strongly and negatively affects his growth. 

Moreover, educational poverty is a multi-dimensional phenomenon, 
which also involve other targets and objectives of the 2030 Agenda. The dis- 
advantage of children and young people is often influenced by the socio-eco- 
nomic family situation, by material factors that penalize good growth (for 
example the region of residence), from inequality of opportunity that is per- 
petuated from generation to generation, in some places and some families. 

The empirical measurement of educational poverty will therefore be stud- 
led through the levels of literacy and numerical competence, knowledge of the 
English language, early school leaving and educational qualifications, trying 
to disaggregate information in a widespread manner to bring out inequalities. 


L'Agenda per lo sviluppo sostenibile del 2030 adottata dall’Assemblea 
generale dell'ONU (risoluzione A7RES/70/1, New York) si basa su 17 Obiet- 
tivi di sviluppo sostenibile (Sustanaible Development Goals) con l'obiettivo 
di porre fine alla povertà, proteggere il pianeta e garantire la prosperità per 
tutti entro il 2030. I 17 Goal sono articolati in 169 target e 244 indicatori. 
Il Goal 4 "Istruzione di qualità per tutti" dell'Agenda 2030 si occupa del 
tema della istruzione di qualità, fattore rilevante per migliorare la vita delle 
persone e rendere attuabile uno sviluppo sostenibile. I target da monitora- 
re riguardano diverse dimensioni: l'accesso per tutti all’istruzione di ogni 
ordine e grado (scuola dell'infanzia, primaria, secondaria e terziaria), la 
qualità dell'istruzione impartita, il possesso delle conoscenze e delle com- 
petenze per l'occupazione e per lo sviluppo sostenibile; l'eliminazione delle 
disparità di genere nell istruzione e la parità di accesso per i piu vulnerabi- 
li; il monitoraggio delle strutture scolastiche, in modo che siano adatte alle 
esigenze di tutti. 

Il monitoraggio del Goal 4 per il nostro Paese si concentra sul percorso 
formativo delle persone dall'accesso alle attività di apprendimento a cinque 
anni, proseguendo con l'inclusione nei livelli di istruzione secondaria e ter- 
ziaria, monitorando le competenze e conoscenze apprese. 

Il presente lavoro mostra come l'integrazione di dati da varie fonti (dati 
INVALSI sulle competenze, dati da indagine da fonte ISTAT e dati da MIUR) 
possa dare un quadro globale della situazione del paese Italia; e come 
dall'aggregazione dei dati, dall analisi di ogni singolo indicatore e dal coin- 
volgimento di altri target e obiettivi dell'Agenda 2030, possano scaturire 
informazioni sostanziali sulle disuguaglianze. 

Ogni bambino e ragazzo ha diritto ad apprendere, formarsi, sviluppare 
le sue capacità, le sue competenze e le sue aspirazioni nel modo piu proficuo 
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possibile e con le migliori opportunità; quando questo diritto non é garan- 
tito, il minore si trova in una condizione di povertà educativa e sconta una 
mancanza di opportunità, che incide fortemente e negativamente sulla sua 
crescita. La povertà educativa é un fenomeno che attiene a piu dimensioni. 
Lo svantaggio dei bambini e dei ragazzi é spesso influenzato dalla situazione 
socio-economica familiare, da fattori materiali che penalizzano la buona 
crescita (ad esempio la regione di residenza), da disuguaglianza di oppor- 
tunità che si perpetua di generazione in generazione, in particolari luoghi e 
in particolari famiglie. 

La misurazione empirica della povertà educativa sarà, quindi, studiata 
attraverso i livelli di competenza linguistica e matematica, la conoscenza 
della lingua inglese, l'abbandono scolastico e le qualificazioni formative ac- 
quisite, cercando di disaggregare l'informazione in maniera capillare per 
far emergere le disuguaglianze. 


1. Introduction 


The 2030 Agenda for sustainable development adopted from the UN-As- 
sembly General (UN Resolution A7RES/70/1, New York) 1s built on 17 Sus- 
tainable Development Goals (SDGs) with the aim to end poverty, protect the 
planet and ensure prosperity for all by 2030. The 17 goals are articulated in 
169 targets and 244 indicators (232 of which are different) for monitoring 
of the Goals. The Goals of the 2030 Agenda refer to social, economic and 
environmental development, which need to be considered with an integrated 
approach that safeguards the planet and guarantees the well-being of people 
and an equitable distribution of development over time. 

The present work shows how the integration of data from various sources 
(INVALSI data, data from ISTAT survey and data from MIUR) can give a 
global picture of the situation in Italy; and how from the aggregation of data, 
from the analysis of each single indicator and from the involvement of other 
targets and objectives of the 2030 Agenda, substantial information on ine- 
qualities in education and on educational poverty can arise. 


2. Purpose of Goal 4: the word-cloud 


The Goal 4 of Agenda 2030 is the goal of “Quality of education". Quality 
education and lifelong learning opportunities for all are central to ensuring 
a full and productive life to all individuals and to the realization of sustaina- 
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ble development. The Targets to be monitored in this goal concern different 
dimensions: access for all to education of all levels (primary, secondary and 
tertiary), a good quality of education; the elimination of gender disparities 
in education and equal access for the most vulnerable; monitoring of school 
facilities, so that they are suitable for everyone’s needs. 
Without going into target detail, the word cloud shows that education 
(words that appear larger are most cited in the 2030 Agenda): 
— must be inclusive (no left leave behind) and be available for boys, girls, 
disabled people, youths, children, vulnerable people; 
— has to be of quality, accessible to all, to enhance skills and to promote 
culture; 
— must increase equality, promote inclusion and sustainability. 


ec et -— 
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EA V gu, devel oe 
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Fig. 1 — Goal 4 word-cloud of Agenda 2030 


3. Dissemination of data in Italy 


To meet the global and national information demand different compo- 
nents of the Italian National Institute of Statistics (ISTAT) worked together: 
an integrated effort to enhance statistical information framework, thematic 
and methodological investments, interinstitutional synergies. 

In Italy, ISTAT proposes, every 6 months, the updating of the indicators 
together with the analysis of their trends, and interconnections, giving a sta- 
tistic framework for monitoring progress towards Sustainable Development. 
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In the Second report of ISTAT, published in 2019, 273 statistical meas- 
ures are available for 123 SDGs indicators (ISTAT, 2019). For the Goal 4 
on “Quality of education”, ISTAT releases 36 statistical measures related to 
eight Targets. 

Goal 4 monitoring “Quality education for all” for Italy focuses on the 
training path of people from access to learning activities at five years', con- 
tinuing with inclusion in secondary and tertiary education levels’, monitor- 
ing skills and learned knowledge’. 


4. Educational poverty and SDGs 


What is meant when we talk about educational poverty? How can the ob- 
jectives of Agenda 2030 help us to learn about this phenomenon? The univer- 
sal principle is that every child and every young person has the right to learn, 
to train, to develop his or her own skills and competences and aspirations in 
the most profitable way possible and with the best opportunities; when this 
right 1s not guaranteed, the child is in a state of educational poverty. 

Usually educational poverty is accompanied by a lack of opportunities, 
which strongly and negatively affects the growth of the child. One can also 
speak of “ex ante inequality" which looks at how different the circumstances 
unintentionally inherited or faced by individuals are and which influence 
their economic performance (Stiglitz et al., 2018). 

The educational disadvantage of children and young people 1s often in- 
fluenced by the family socio-economic situation, by material factors that 
penalize good growth (e.g. the region of residence), by the inequality of op- 
portunity that is perpetuated from generation to generation, in some places 
and in some families. 


! Target 4.2 *By 2030, ensure that all girls and boys have access to quality early childhood 
development, care and pre-primary education so that they are ready for primary education". 

? Target 4.3 “By 2030, ensure equal access for all women and men to affordable and qual- 
ity technical, vocational and tertiary education, including university", Target 4.5 *By 2030, 
eliminate gender disparities in education and ensure equal access to all levels of education and 
vocational training for the vulnerable, including persons with disabilities, indigenous peoples 
and children in vulnerable situations" and Target 4.6 *By 2030, ensure that all youth and a 
substantial proportion of adults, both men and women, achieve literacy and numeracy”. 

? Target 4.1 *By 2030, ensure that all girls and boys complete free, equitable and quality 
primary and secondary education leading to relevant and effective learning outcomes" and 
Target 4.4 “By 2030, substantially increase the number of youth and adults who have relevant 
skills, including technical and vocational skills, for employment, decent jobs and entrepre- 
neurship". 
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In addition, educational poverty is a multidimensional phenomenon, 
which also involves other objectives and targets of Agenda 2030. 

The reading of the interconnections and interactions between themes, ar- 
eas and objectives is useful to trace the paths of analysis to investigate com- 
plex phenomena such as educational poverty". 

Goal 4 “Quality of education” is very connected with Goal 3 “Good 
health and well-being”, Goal 5 “Gender inequality”, Goal 8 “Decent work 
and economic growth” and Goal | on poverty (Fig. 2). 
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Fig. 2 — The wheel of Goal 4 


4 The network is built by looking at the UN metadata (https://unstats.un.org/sdgs/): for 
each indicator the interrelationships with another indicator are indicated. These links have 
been represented in matrices and visualized by charts. 
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5. Educational poverty through data: the situation in Italy 


Educational poverty involves several indicators of the 2030 Agenda: 
SDG 4.1.1 — “Proportion of children and young people: (a) in grades 2/3; (b) 
at the end of primary; and (c) at the end of lower secondary achieving at least 
a minimum proficiency level in (i) reading and (ii) mathematics, by sex” and 
SDG 4.3.1 “Participation rate of youth and adults in formal and non-formal 
education and training in the previous 12 months, by sex”. 

SDG 4.1.1 will be studied through the levels of literacy and numerical 
competence; SDG 4.3.1 with the indicator “Early leavers from education and 
training — ELET (aged 18-24)”. 


5.1. The INVALSI data 


In Italy, in 2019, the share of children enrolled in the third year of lower 
secondary education who are low performers in literacy skills is 34.4%, in 
mathematics 38.7%, slightly better than 2018. 

There are many territorial, gender and citizenship differences that often 
fuel inequalities, with regard to access to educational opportunities. The low- 
est percentages of students (corresponding to the best performance in the ter- 
ritory) with low levels of literacy are recorded in Marche (25.6%), in Veneto 
(26.8%), in Valle d’ Aosta (27.7%) and in Friuli-Venezia Giulia (27.8%). 

For numerical skills, the percentage of inadequacy is lower in the prov- 
ince of Trento (23.7%), in Friuli-Venezia Giulia (25.4%), Veneto (26.4%) 
and Toscana (28.5%). Calabria, with 49.5% of lows performers in literacy, 
Sicilia (46.8%) and Campania (44.8%) are the regions where the levels of 
students with low literacy skills are the highest; also for numerical levels 
skills of third year pupils in lower secondary schools degree, these regions 
maintain the highest levels of insufficiency, Calabria and Sicilia with 59.5% 
and Sicilia with 56.5%. 

Compared to females, a higher percentage of boys is below the required 
level in literacy skills (39.1%, against 29.4% of females), while for numer- 
ical skills the situation is reversed: 39.9% of the girls does not reach the 
sufficiency, against 37.5% of the boys. 


> Percentage of people aged 18-24 years who have achieved only lower secondary edu- 
cation (ISCED level 2) and are not included in education or training program on total people 
aged 18-24 years. 
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Fig. 3 — Share of students in grade 8 performing below the baseline level of proficien- 
cy in literacy competence by region, gender and nationality — Years 2018 and 2019 
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Fig. 4 — Share of students in grade 8 performing below the baseline level of proficien- 
cy in numeracy competence by region, gender and nationality — Years 2018 and 2019 
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Fig. 6 — Share of students in grade 10 performing below the baseline level of proficien- 


cy in numeracy competence 


An evidence of the inequalities which exist due to different access op- 
portunities is the strong difference in the competences of the children born 
in Italy from Italian people (native), compared to the first and second gener- 
ation foreigners (respectively children born abroad from immigrant parents 
and children born in Italy to immigrant parents). 63.9% of children who were 
not born in our country do not reach the literacy sufficiency and 57.6% the 
numerical one. A measure of the integration capacity of foreign students in the 
school system is the difference in results between the first and second genera- 
tion of immigrants and the natives. Youngsters born in Italy from foreign par- 
ents behave slightly better, both in terms of literacy (50.9% not sufficient) and 
in mathematic skills (45.5%). The percentage of Italian students who do not 
reach sufficiency is 31.5% for literacy skills and 36.9% for numerical skills. 

Among the second year students in upper secondary school, 30.4% do 
not reach a sufficient level in terms of literacy and 37.8% in terms of mathe- 
matics. Regional differences are wide. Students in the North are less lacking 
in both language and mathematical skills, the Centre’s students are placed 
in the national average levels and the students in the South and Islands have 
particular shortcomings especially in mathematics (more than half of the stu- 
dents from Campania, 55.5%, Sicilia, 57.1%, Calabria, 57.7% and Sardegna, 
60.5% are insufficient). The difference between girls and boys is wide for 
language skills (34.3% of male students do not reach the required level com- 
pared to 26.3% of female students), and less strong for mathematical skills in 
favor of boys (33.5% of males against 42.2% of females). 

Upper secondary school is divided into three main channels: high schools, 
technical colleges and vocational colleges. The result at national level by 
type of school is very different, with 16% of high school students not achiev- 
ing a sufficient level in Literacy and 27.8% in Mathematics; among those at- 
tending technical schools, 37.6% and 37.9% are insufficient in Literacy and 
Mathematics respectively; among students in vocational schools, the results 
are very discouraging, with 66.7% not achieving a sufficient level of literacy 
and 73.4% of numerical skills. 


5.2. The ELET indicator 


In Italy, the drop from 20% in 2008 to 13.8% in 2016 of the indicator on 
early school leaving represents a significant progress but, in 2017 and 2018, 
the rate of early leavers has risen to 14.5% with clear territorial differences. 

In 2018, the rate of early leaving, stable for the South and the Centre, but 
on the rise for the North-West area and the Islands, remained, however, very 
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high in the Islands and in the South 18.8%, against 10.6% in the North-East, 
10.7% in the Centre and 13.3% of the North-West. 
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Fig. 7 — Share of ELET (Early leavers from education and training) by geographic 
region — Years 2004-2018 


The difference between the girls, more involved in the education system, 
and the boys is significant: in 2018, 16.5% of boys dropped out of school 
and were not part of a training course against 12.3% of girls (Baldazzi and 
Cascioli, 2019). 

The proportion of 18-24 year olds Italian and foreign citizenship who 
are not included in an education and training is relevant: among the natives, 
12.3% left school and education, among foreigners 37.6%. 

Among the regions, the Trento Province, Umbria, Abruzzo and Friu- 
li-Venezia Giulia showed values below 10%, while in Calabria, Sicilia and 
Sardegna the values exceeded 20%. 


6. An example of interconnections and the interactions between 
themes in Goal 4 


Within each Goal it is possible to make a first analysis of the intercon- 
nections and interaction between the indicators. Principal component anal- 
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ysis (PCA) is a statistical method that uses an orthogonal transformation 
to convert a set of observations of variables into a set of values of linearly 
uncorrelated variables called principal components, with the first principal 
component has the largest possible variance and each succeeding component 
in turn has the highest variance. Components are orthogonal. 

The variables of Goal 4 are 11, calculated on 21 regions representing the 
analysis units. 
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Fig. 6 — PCA for variables of Goal 4 — Year 2019 


The first component (represented by the horizontal axis) represents “in- 
tegration and success in the school system". In the negative part of the axis 
there are the digital competence variable and the percentage of people with 
a tertiary level of ISCED, while on the positive semi-axis there are the low 
competence variables in literacy and numeracy, and the ELET variable. 

The second component (represented by the vertical axis) represents the 
"advancement in skills (also extracurricular)". In the negative part of the 
axis there are the variables of the low competence in Listening English and 
Reading English, while on the positive semi-axis there are the percentage of 
people doing lifelong learning. 
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The regions with the best profiles in Goal 4 are the province of Tren- 
to, Friuli-Venezia Giulia, Emilia-Romagna, Valle d’Aosta, the province of 
Bolzano, Liguria and Lazio, which are in the fourth quadrant. Lifelong learn- 
ing is highly developed in these regions. 

Other regions with positive values in many of examined variables are 
Lombardia, Marche, Toscana, Piemonte, Umbria and Veneto which are posi- 
tioned in the third quadrant. 

More difficult the position of the regions positioned in the second quad- 
rant: Abruzzo, Molise, Basilicata, Puglia and Campania. Sicilia, Calabria 
and Sardegna, which are positioned in the first quadrant, are the regions most 
affected by some of the goal 4 indicators. It is precisely in these 8 regions 
that the highest percentages of children at risk of educational poverty, often 
associated with forms of economic poverty and severe deprivation, are con- 
centrated (Openpolis, 2019; Save the children, 2019). 


7. Conclusion 


The wide database of SDGs and the continuous updating of the data al- 
lows to analyze in depth the situation of each Goal of the 2030 Agenda. In 
the case of Goal 4, integration of data from various sources (INVALSI data, 
OECD-PISA data, data from ISTAT survey and data from MIUR) gave a 
global picture of the situation of “Quality of education" in Italy. 

The wideness and granularity of the indicators disseminated by Agenda 
2030, its sharing at global level, allows the study of complex and multidi- 
mensional phenomena such as educational poverty. 

There are many territorial, gender and citizenship differences that often 
fuel inequalities, with regard to access to educational opportunities. 

Regional differences are wide. Students in the North are less lacking 
both in Linguistic and in Mathematics skills, the students of the Center are 
positioned in the national average levels and the students of the South have 
particular deficiencies especially in Mathematics. 

An evidence of the inequalities which exist due to different access oppor- 
tunities is the strong difference in the competences of the children born in It- 
aly from Italian (native) parents, compared to the first and second generation 
foreigners (respectively children born abroad from immigrant parents and 
children born in Italy to immigrant parents). 

A first analysis of the interconnections and interaction of indicators of 
Goal 4 shows two principal components. The first component represents 
"integration and success in the school system"; the second component rep- 


122 


ISBN 9788835131076 


resents the “advancement in skills (also extracurricular)”. The regions with 
the best profiles in Goal 4 are the province of Trento, Friuli-Venezia Giulia, 
Emilia-Romagna, Valle d’ Aosta, the province of Bolzano, Liguria and Lazio. 
Sicilia, Calabria and Sardegna are the regions most affected by some of the 
goal 4 indicators. It is precisely in these 3 regions that the highest percentag- 
es of children at risk of educational poverty, often associated with forms of 
economic poverty and severe deprivation, are concentrated. 

The extension of the network of connections with the other Goals and the 
involvement of other targets and objectives of the 2030 Agenda will give an 
even more complete picture of the situation in Italy regarding educational 
poverty and quality education. 
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Data are precious resources and as such an asset of the com- 
munity. Collected, ordered, studied data become a tool that al- 
lows us to open debates and make most useful decisions. The 
INVALSI Statistical Service followed up, on this premise, with the 
organization of the IV Seminar "INVALSI data: a tool for teach- 
ing and scientific research". The Institute's databases, in fact, 
allow researchers and teachers to investigate in depth the theme 
of education in schools from different points of view and the 
event allows them to participate in a lively confrontation on the 
subject. The authors of the contributions collected in this vol- 
ume investigate in depth the characteristics of students, schools 
and society. As a Statistical Service we hope that the reading of 
the volume confirms what has been written on the importance 
of data and their use, which will allow an ever-growing audience 
to enrich their knowledge on the subject of education and be a 
means of giving life to always new ideas for further reflection. 
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